Regularization is a set of techniques used to prevent a model from overfitting, by discouraging it from becoming too complex or relying too heavily on any single feature.
What it means in plain English
Left unchecked, a model may fit its training data too closely, memorising noise instead of learning general patterns. Regularization pushes back by adding a penalty for complexity during training, nudging the model toward simpler, more general solutions. The effect is a model that performs better on new, unseen data — which is what actually matters.
It is one of the core tools for building models that generalise well.
A simple example
When a model starts memorising quirks of the training data, regularization gently penalises overly large or complex patterns, steering it back toward the general rule — much like reminding a student to understand the concept rather than memorise the exact practice questions.
Why it matters
Regularization is essential for building models that work in the real world rather than just on their training data. It directly addresses overfitting, one of machine learning’s central problems, and is a routine part of training good models.
Related terms
- Overfitting — the problem regularization prevents.
- Dropout — a specific regularization method for neural networks.
- Hyperparameter — regularization strength is often one.
Frequently asked questions
What is regularization?
Regularization is a set of techniques that discourage a model from becoming too complex, helping prevent overfitting so it generalises better to new data.
What are common regularization methods?
Examples include L1/L2 penalties on weights, dropout in neural networks, and early stopping during training.
