The hype arc of the 2010s looks like this: self-driving cars by 2018, AGI by 2025, every job automated by 2030. None of it happened on schedule. Most of it is still more complicated than the predictions suggested.
The actual capability arc looks different, and it's worth writing it down from inside the decade rather than waiting for someone to reframe it retrospectively.
What actually changed
Deep learning worked. This sounds obvious now. In 2010 it was not obvious. The deep learning bet that Hinton, LeCun, and Bengio were making for years looked speculative until AlexNet in 2012. ImageNet performance jumped far enough to be undeniable. After that, the field moved fast.
Computer vision crossed a threshold. Not theoretical threshold. Practical threshold: models that can be deployed in real products and do useful work. Image classification, object detection, face recognition. These transitioned from research topics to engineering problems in the middle of this decade. By 2019, if you can define a visual recognition problem precisely, you can probably build a model that handles it to commercial quality.
Transfer learning changed the economics. The ImageNet pretraining plus fine-tuning approach became standard in vision by 2014 or so. The equivalent for NLP arrived at the end of this decade with BERT and the ULMFiT approach. The cost to adapt a model to a new domain dropped dramatically. This is underappreciated. It means the barrier to applied ML work is now mostly about data collection and problem definition, not about training capability from scratch.
Hardware costs collapsed. Training a model that would have required a research lab in 2010 now runs on a GPU you can rent by the hour. The democratization of compute changed who can do ML work. By the end of this decade, a small team with domain expertise and a training budget can build things that would have required a university lab five years ago.
What didn't change
The fundamental limitation of these systems. Deep learning models learn correlations in data. They don't have causal understanding. They don't reason about situations they haven't seen. They fail on distribution shift in ways that humans don't. Ten years of progress hasn't changed this. The capabilities improved enormously and the underlying limitation remains.
The need for labeled data. Transfer learning reduced the amount you need. It didn't eliminate the need. Collecting high-quality labeled data for a specific domain is still expensive and still the binding constraint for most applied ML projects.
The interpretability problem. We build models we can't fully explain. In low-stakes domains this is acceptable. In high-stakes domains it's a genuine unsolved problem. The decade produced impressive benchmark numbers and limited progress on understanding why the models work.
What changed late in the decade that I'm watching into 2020
GPT-2. Language models at scale producing coherent text. This is new. We don't fully know what it means yet. The capabilities feel qualitatively different from everything that came before in NLP. Whether that difference translates to practical utility or represents something more fundamental is an open question.
The attention mechanism and transformer architecture. The paper was 2017. The applications are still landing. BERT in October 2018 showed that transformers generalize across NLP tasks. GPT-2 showed that scaling transformers on generative tasks produces capabilities nobody specifically trained for. The decade ended with a new architecture that will probably define the next several years.
What to expect
Not AGI. Not full automation. More capable tools that require human judgment to deploy well. The pattern of the last decade is that AI capabilities improve, deployment complexity reduces, and the human role shifts toward problem definition and output evaluation rather than implementation.
That's a meaningful shift. It's not the shift the predictions promised.
With gusto, Fatih.