Catastrophic Forgetting is a problem where a neural network, when trained on a new task, abruptly loses much of its ability to perform tasks it previously learned.
What it means in plain English
Unlike humans, who build on what they know, a neural network trained on something new can overwrite the knowledge it already had — sometimes drastically. As it adjusts its weights for the new task, the patterns that supported the old task get disrupted, and its performance on the old task collapses. This makes it hard to teach a model new things continuously without it forgetting the old.
Overcoming it is an active research challenge in building AI that learns continually.
A simple example
A model trained to recognise cats, then retrained only on dogs, may suddenly become much worse at recognising cats — having “forgotten” the earlier skill as it learned the new one.
Why it matters
Catastrophic forgetting is a key obstacle to AI that can learn continuously and accumulate knowledge over time, the way people do. Understanding it explains why models are usually retrained on all their data at once, rather than updated task by task.
Related terms
- Transfer Learning — a related idea of carrying knowledge across tasks.
- Fine-Tuning — where forgetting can occur.
- Neural Network — where the problem arises.
Frequently asked questions
What causes catastrophic forgetting?
When a neural network is trained on new data, the weight updates for the new task can overwrite what it learned before, causing it to “forget” earlier tasks.
How is catastrophic forgetting addressed?
Approaches include rehearsing old data alongside new, protecting important weights (regularisation-based methods), and architectures designed to add new knowledge without overwriting old.
