Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep Learning

Why a model that learns something new tends to lose what it already knew, explained from a single fact: weights are a shared resource. Walks through EWC, replay, and swappable LoRA adapters with equations and interactive figures, then explains why production teams still just mix the old data back in and retrain.

Overcoming catastrophic forgetting in neural networks

Writing on a clay tablet

Picture carving letters into a clay tablet. You carve a second message over the first. The clay is soft, so the new letters come out crisp — and to exactly that extent, the old ones are flattened. Fragments survive, but the sentence is gone.

The same thing happens when you keep training a model. Take a model that summarizes well, then fine-tune it on nothing but medical intake transcripts. The intake responses get dramatically better, and the summarization it used to handle fine falls apart. Not gracefully, either: it can collapse within a few hundred steps. McCloskey and Cohen reported the effect in 1989, and it has been called catastrophic forgetting ever since.

The awkward part is that this is not a bug. The optimizer is doing precisely what it was told. It was just told only one thing: lower the loss on the data in front of you.

Forgetting is a fight over shared resources

Once you remember what a model actually is, the collapse stops being mysterious. Knowledge is not filed away somewhere as a discrete record. The same weight matrices serve summarization and intake notes, English and arithmetic, all at once. Weights are a shared resource.

Gradient descent on new data moves those weights using only the new loss. If the direction that helps the new task happens to hurt the old one, nothing pushes back, because the old loss is never computed. There is no brake — not a weak brake, but no brake at all.

This tug of war is the classic stability–plasticity dilemma. Freeze the weights to protect the past and you can't learn anything new; let them move freely and the past is overwritten. Every continual learning method is a proposal about where to draw that line.

Try it on the figure below. Put the ball in the left basin — the optimum for the old task — and let it roll toward the right basin, the new task. The path necessarily leaves the left basin, and the larger the learning rate, the more violently it departs. That is what forgetting looks like.

Measuring how much was lost

Before fixing anything, decide how to measure it. In continual learning you train on tasks $1, 2, \dots, T$ in sequence, and right after finishing task $t$ you evaluate every task seen so far. Writing $a_{i,t}$ for the accuracy on task $i$ after training through task $t$, forgetting is defined as:

In words, equation (1) averages, over every task, the gap between its best score ever and its score at the end. If you only track mean final accuracy, gains on the newest task mask the losses elsewhere. The drop has to be reported separately.

One more distinction you can't skip: the three scenarios of van de Ven and Tolias. In task-incremental learning, inference time tells you "this is a task 3 question." In domain-incremental, it doesn't, but the kind of question stays fixed. In class-incremental, the set of possible answers itself keeps growing. Difficulty climbs in that order, and a method that shines in the first routinely does nothing in the last.

There are only three families of fix

The literature looks enormous, but methods differ mainly in what they spend to buy stability. You can pin the weights (narrow where they may move), mix in old data (pay in storage and compute), or separate the real estate (pay in parameters). We'll take them in that order — and then look at why production ends up somewhere else entirely.