Why has AI improved so fast?

Last updated: July 2026

In 2019, AI writing was a novelty that produced barely coherent paragraphs. A few years later it was passing exams and writing working software. That's an astonishing jump in a short time. What actually caused it? The answer is less about a single genius invention than most people think.

It's easy to assume the leap in AI came from some brilliant breakthrough in making machines think. The real story is stranger and, in a way, more unsettling. A large part of the progress came from a simple, almost brute-force discovery: if you make these systems bigger and feed them more, they get predictably better. Not randomly better. Predictably better, in a way you can chart in advance.

That discovery has a name, and understanding it explains most of what's happened. But let's build up to it, because two other ingredients had to come first.

Ingredient one: the architecture that could scale

Before 2017, the software used for language processing had a bottleneck. It mostly read text one word at a time, in order, which made it slow to train and forgetful over long passages. You couldn't easily make it much bigger, because bigger just meant slower.

Then in 2017 a group of Google researchers published a design called the transformer. Its key feature was that it could process all the words in a passage at once, in parallel, rather than plodding through them one by one.¹ This sounds like a technical detail. It was actually the thing that unlocked everything else, because "can be run in parallel" is the same as "can be scaled up on lots of computer chips at the same time."

One analysis of the field's history put it bluntly: the transformer created a highly parallel, scalable architecture that improved as you made it larger. That property was the revolution.²

Ingredient two: the discovery of scaling laws

Once you have an architecture that can grow, the question becomes: is it worth growing? Does a bigger model actually get better, or does it just get more expensive?

In 2020, researchers at OpenAI published a landmark paper called "Scaling Laws for Neural Language Models" that answered this precisely.³ They trained many models of different sizes and measured how performance changed. What they found was a strikingly smooth, predictable relationship. Performance improved reliably as you increased three things together:

• The size of the model (roughly, how many internal settings, called parameters, it has)
• The amount of data it trains on
• The amount of computing power used to train it

The relationship followed what's known as a power law, which just means the improvement was smooth and predictable across an enormous range of scales.⁴ This was the pivotal finding. It meant that improving AI wasn't a matter of waiting for the next clever idea. You could improve it on schedule, simply by scaling everything up, and you could predict roughly how much better it would get before you even started.

Chip-maker NVIDIA describes the three elements, data, model size, and compute, as deeply interrelated: feed larger models more data and performance improves, but only if you also scale up the computing power to handle the load.⁵ That last requirement is why the AI boom and the demand for enormous computing hardware happened at the same time.

Why this made progress feel so sudden

Here's the thing about a smooth, predictable trend: it can still feel like an explosion from the outside.

Companies realised that scaling worked, and a race began to build bigger and bigger models. Each new generation was trained with far more data and computing power than the last. Because the scaling relationship held, each generation was reliably more capable. From the inside it was a steady climb up a known curve. From the outside, where the public only saw the released products, it looked like AI leapt from "amusing toy" to "writes better than most people" almost overnight.

A refinement arrived in 2022, sometimes called Chinchilla scaling, which found that many models had been made too big and undertrained. The better recipe was to balance model size and data more evenly, roughly doubling the training data each time you double the model size.⁶ This made the next round of models more efficient, squeezing more capability out of the same resources.

Does scaling go on forever? No.

It's tempting to conclude that AI will just keep improving indefinitely as long as someone keeps building bigger computers. The reality is more nuanced, and researchers have been clear about the limits.

The improvements from scaling show diminishing returns. Each doubling of resources brings a smaller gain than the last. Eventually you reach what's called the compute-efficient frontier, a point where adding more resources gives so little extra performance that brute-force scaling stops being practical.⁷ You also eventually run low on the two other ingredients: high-quality training data is finite, and computing power costs real money and energy.

This is why the most recent progress has come from new directions, not just bigger models. One important shift, seen in reasoning models from 2024 onward, was giving AI more time to "think" when answering a hard question rather than only making the model itself larger.⁸ Letting a model work through a problem step by step before answering turned out to be another lever to pull, separate from raw size.

The short version

AI improved so fast because three things lined up. First, the transformer arrived in 2017 and gave researchers an architecture that could be scaled up efficiently. Second, the 2020 scaling laws revealed that making models bigger, with more data and more compute, improved them predictably, which turned progress into something you could plan and fund on purpose. Third, companies poured enormous resources into that scaling, and the results arrived on schedule.

What looked like a series of miracles was, underneath, a fairly steady march up a curve that researchers had already mapped. That's arguably the most remarkable part. The great surprise of modern AI wasn't a single spark of machine genius. It was the discovery that intelligence-like behaviour could be summoned, in large part, just by scaling simple ingredients up far enough. Whether that march continues, or hits its limits, is the open question the field is wrestling with right now.

Sources

1. Vaswani et al. "Attention Is All You Need" (2017), the paper introducing the transformer architecture.
2. "AI Foundations: Transformers, Pre-Training, and Scaling," Fabricated Knowledge (2023), fabricatedknowledge.com
3. Kaplan et al. "Scaling Laws for Neural Language Models," OpenAI (2020).
4. Haider, "Scaling Laws in AI: The Mathematical Foundation," Medium (2025), medium.com
5. "How Scaling Laws Drive Smarter, More Powerful AI," NVIDIA Blog (2025), blogs.nvidia.com
6. "Demystify Transformers: A Guide to Scaling Laws," Sage AI, Medium (2024), medium.com
7. "AI Model Scaling Isn't Over: It's Entering a New Era," AI Business (2025), aibusiness.com
8. "Neural scaling law" (on 2024 test-time compute and o1), Wikipedia, en.wikipedia.org