Learning Rate is a hyperparameter that controls how big a step a model takes when adjusting its parameters during training. It is one of the most important settings to get right.
What it means in plain English
Each time a model learns from its errors, it adjusts its parameters. The learning rate controls the size of that adjustment. Too large, and the model overshoots and training becomes unstable, never settling on good values. Too small, and training is painfully slow and may get stuck. Finding a good learning rate is often the single most impactful tuning decision in training a model.
Many modern training methods adjust the learning rate automatically over the course of training.
A simple example
Picture searching for the lowest point in a valley. A large learning rate means giant leaps that might jump right over the bottom; a tiny one means baby steps that take forever to get there. The right learning rate strikes the balance.
Why it matters
The learning rate has an outsized effect on whether training succeeds at all. It is frequently the first hyperparameter practitioners tune, and understanding it explains why the same model can train beautifully or fail completely depending on one number.
Related terms
- Gradient Descent — the process the learning rate governs.
- Hyperparameter — the category learning rate belongs to.
- Loss Function — what a good learning rate helps minimise.
Frequently asked questions
What is the learning rate?
It is a hyperparameter that controls how large a step the model takes when updating its parameters during training — how fast it “learns” from each batch.
What happens if the learning rate is wrong?
Too high and training can overshoot or become unstable; too low and it trains very slowly or gets stuck. Finding a good learning rate is key to effective training.
