forgetting

Definition

Forgetting is the loss of previously learned capability caused by subsequent training. Mechanically it is overwriting: the weights that encoded the old task get reshaped by gradients from the new one. It is the central obstacle in continual learning, and the reason fine-tuning a deployed model can quietly degrade behavior nobody thought to retest. Unlike human forgetting it is not driven by elapsed time — an untouched checkpoint forgets nothing.

Explainers using this term

• Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep LearningOvercoming catastrophic forgetting in neural networks