Skip to content

Self-improving AI

How Do We Reach Continuous Learning?

Every AI model you’ve used so far was finished before you met it.

  • No. 138
A restrained 3D scene on a dark navy field: four separate frosted glass blocks labelled static models stand in a row, and from the last one a single unbroken orange ribbon labelled keep learning rises to the right.
In this article8 sections

Every AI model you’ve used so far was finished before you met it. It was trained, tested, frozen and released, and from that point on it stopped learning. When it got better, it was because a new version replaced it.

Self-improving AI is one of the big talking points at the moment, and the version I’m focused on is the one that actually ends the current model release pattern we’re seeing. Instead of static models and release cycles, we’d have models that keep learning and improving themselves while they’re in use. I expect us to head quickly in that direction during 2027.

But I wanted to first take a step back and understand what’s pushing us there, what the trends suggest and what’s still in the way.

The Recipe That Got Us Here

For most of the last decade, progress in AI followed a fairly reliable recipe. In 2020, researchers at OpenAI published what became known as the scaling laws. They showed that a language model’s error falls in a predictable way as you increase three things: the computing power used to train it, the amount of data it learns from, and the number of parameters inside it.

So the industry made models bigger. The largest language models went from hundreds of millions of parameters in 2018 to 175 billion in 2020, and the frontier has kept climbing since, with today’s largest estimated to run into the trillions.

Size brought more than lower error rates. Larger models could pick up a new task from a handful of examples in the prompt, without being retrained for it. And some abilities only appeared past a certain size. A 2022 paper on emergent abilities showed models performing no better than chance at tasks like modular arithmetic or unscrambling words, until they reached a certain scale and could suddenly do them. Step-by-step reasoning, which I wrote about in April, followed the same pattern: small models got nothing from it and larger ones got a lot. If making the model bigger keeps producing abilities nobody predicted, you keep making it bigger.

But size is only half of how today’s assistants were built. Pre-training, where the model learns to predict the next word across a huge amount of text, gives you something that knows a lot about the world but doesn’t know how to follow an instruction, or what people consider helpful or safe. Everything after that leans on people. The model is fine-tuned on carefully selected, high-quality text, then on question-and-answer examples that teach it to follow instructions, and then on human judgements about which of its answers are better. That last step is reinforcement learning from human feedback, and it did a lot to turn a capable model into one people wanted to talk to. Even with all of that, many researchers admit that getting models to follow human goals and values is still an open problem.

Why It’s Running Out

Two things are changing. The first is that scaling the old way is giving less back. We’ve started hitting saturation point. More compute, more data and bigger models still help, but each step costs far more for a smaller gain.

The second is data. The scaling laws assume there’s always more good text to learn from, and there isn’t. Epoch AI estimated in 2024 that, if current trends continue, models will be trained on datasets about the size of all the public human-written text in existence somewhere between 2026 and 2032. The later stages of training are worse off still, because they depend on human-made examples and judgements that are slow and expensive to produce.

For me, those two together are the main reason the field is moving towards self-improvement. When the old recipe slows down and the ingredients run short, you need models that can produce their own progress.

Where the New Data Comes From

If the internet has largely been read, new data has to come from somewhere else, and I see three main sources.

The first is the use of the models themselves. Every conversation, correction and abandoned answer says something about what worked and what didn’t, and a huge number of people now use these tools every day. That makes usage the biggest new source of data there is.

The second is synthetic data, where models create training material for other models, or for themselves.

The third is the one I find most interesting: new connections that humans haven’t already made. A model has read across disciplines in a way no single person ever could. It can put a finding from one field next to a problem in another and propose a link that nobody has written down. If that link holds, it’s new knowledge, and new knowledge is exactly what a model needs once it has run out of human text to learn from.

So how do you know a link holds, and isn’t a confident hallucination? The same way we know with people: empirical research checks it. A scientist’s idea isn’t a discovery until it’s been tested and replicated, with all the nuance that “proven” carries, and a model’s idea shouldn’t be held to a lower standard.

Uneven by Nature

That standard has a consequence. A self-improving model can only learn as fast as something can confirm it got things right. In code and maths a machine can check the answer in seconds, because the tests pass or they don’t and the proof holds or it doesn’t. In medicine, materials science or economics, proof means experiments and trials, and often that takes years.

So I’d expect self-improvement to move fast in some areas and slowly in others. That sounds like a weakness until you look at how people learn. We pick things up quickly where the feedback is quick, and slowly where it takes years, like finding out whether a career move was the right one. Nobody expects human learning to be even, and I don’t think we should expect it from AI either.

What’s Still in the Way

There are three main technical blockers.

The first is catastrophic forgetting. When a neural network learns something new, it tends to overwrite what it already knew. Researchers described the problem in 1989, and it’s still the main reason models are trained in large batches and then frozen. There’s been progress, such as methods that protect the parts of a network that matter most to earlier tasks, but as far as I’m aware, nobody has yet shown a large model learning every day without losing what it learned last month.

The second is model collapse. A 2024 paper in Nature showed that models trained over and over on their own output get worse with each generation. The rare and unusual cases go first, and the model drifts towards a blander average of itself. Synthetic data only helps if something stops that drift, which brings us back to checking against the real world.

The third is whose data it is. If usage is the biggest source of new data, then the people using the models are training them. That raises questions about consent and privacy, and about whose habits and mistakes the model ends up learning. A model that learns from its users will reflect them, for better and for worse.

Then there’s the blocker that I think is most important of all.

Governing Something That Never Stands Still

A static model gives everyone a checkpoint. The lab tests it and publishes what it found, a business approves a specific version, and you get used to how it behaves. If the model keeps learning, the one approved on Monday isn’t the one running on Friday, and every approval process built around fixed versions stops working.

How do you govern something that never stands still? Other fields have already had to answer that.

Software got there first. It used to deploy in large, signed-off releases, and plenty of teams now release changes many times a day. Governance didn’t disappear when the release gate did. It moved into controls that run all the time: automated tests on every change, releasing to a small group before everyone, monitoring what happens in production, and being able to roll back when something goes wrong.

Medical regulators already approve software that changes after approval. In December 2024, the US Food and Drug Administration finalised guidance on Predetermined Change Control Plans for AI-enabled medical devices. The manufacturer sets out in advance what the algorithm is allowed to change, how each change will be validated and how its impact will be assessed, and changes inside that plan don’t need a fresh approval. The regulator approves the limits of change rather than a frozen product.

Banks have treated models as things that drift since 2011, when US regulators issued their guidance on model risk management, known as SR 11-7. It expects an inventory of models, ongoing monitoring and regular checks of outcomes, so approval is never treated as permanent.

Professions have always worked this way. Nobody freezes a surgeon’s knowledge on the day they qualify. We govern the person rather than what they knew on one particular day, through a licence that sets the scope of what they may do, recurrent checks throughout their career (airline pilots keep going back into the simulator), and closer supervision where the risk is higher.

And nature solved it long before any of us. The immune system learns all the time and gets better at recognising threats it has met before. What keeps it safe is a fixed definition of “self”. Immune cells that would attack the body’s own tissue are weeded out as they develop, and other mechanisms keep checking afterwards. When that boundary fails, you get autoimmune disease, essentially a learning system turning on the body it’s meant to protect.

Put those side by side and they arrive at the same answer. None of them governs a snapshot. Each one governs a core that doesn’t change (the immune system’s sense of self, or a model’s values), a boundary on what’s allowed to change, and continuous monitoring with a way to undo a change that goes wrong.

I think that’s where AI governance has to go. It’s also where AI is harder than any of these examples. We can watch a pilot fly and test a medical device against its plan, but we still can’t reliably see inside a model to tell what changed, so for a while monitoring will show what a model does without explaining why. Evolution also has no way to undo a change, and we do. Keeping the ability to roll a model back to a version we trusted might be the most valuable control we have.

Living With a Model That Learns

For anyone who uses AI seriously, at work or at home, the change is bigger than it sounds. Today you can form a view of a tool and that view holds until the next version. With a model that keeps learning, trust becomes something you keep checking rather than something you decide once. The assistant that handled your work well this month might handle it differently next month, and you won’t necessarily get a release note telling you.

It also changes your part in it. If usage is the new training data, then how you use these tools shapes what they become: what you correct, what you push back on, what you accept without checking. That’s as true for a business running its own processes through a model as it is for someone using one to think through a decision about their health or their money.

For organisations, the precedents above aren’t theoretical. If your approval process assumes a model version stays fixed, it’s worth asking what it would look like if the version never stopped changing.

Will 2027 be the year continuous learning properly arrives? Realistically, I have no idea. What I’m confident about is the direction. The old recipe is slowing, the data is running out, and the most promising way forward is models that learn from use, from their own output and from connections we haven’t made yet. The blockers are real, and I think the hardest of them is governance: working out how we trust something that’s never finished.

Further Reading

  • Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models. OpenAI. arXiv:2001.08361.
  • Wei, J. et al. (2022). Emergent Abilities of Large Language Models. Transactions on Machine Learning Research.
  • Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022.
  • Villalobos, P. et al. (2024). Will we run out of data? Limits of LLM scaling based on human-generated data. Epoch AI.
  • McCloskey, M. and Cohen, N. J. (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation, vol. 24.
  • Kirkpatrick, J. et al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences.
  • Shumailov, I. et al. (2024). AI models collapse when trained on recursively generated data. Nature, 631.
  • US Food and Drug Administration (2024). Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions. Final guidance, December 2024.
  • Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency (2011). SR 11-7: Supervisory Guidance on Model Risk Management.
  • Nowak, O. (2026). If AI just predicts the next word, how does it think? The Digital Iceberg.

If this was useful, get the next one by email.

New articles on how AI works and what it changes, sent to your inbox when they are published.

One email for each new article, usually once or twice a month. Unsubscribe at any time.