Hyperparameter is a configuration setting chosen before training that controls how a model learns — as opposed to a parameter, which the model learns during training.
What it means in plain English
Hyperparameters are the knobs a practitioner sets up front. They govern the learning process itself: how fast the model learns, how large the network is, how long training runs. Unlike parameters, they are not learned from data — they are decided by the person building the model, often through experimentation. Choosing them well can be the difference between a model that trains successfully and one that fails.
Finding good hyperparameters, sometimes called hyperparameter tuning, is a normal and important part of building models.
A simple example
The learning rate — how big a step the model takes when adjusting itself — is a hyperparameter. Set it too high and training becomes unstable; too low and it takes forever. The practitioner picks it, then adjusts based on results.
Why it matters
Hyperparameters shape how effectively a model learns, and tuning them is a routine part of machine learning practice. Understanding the distinction between hyperparameters (set by humans) and parameters (learned by the model) clarifies who controls what in the training process.
Related terms
- Parameter — values learned during training, in contrast.
- Learning Rate — a common and important hyperparameter.
- Epoch — another setting that governs training.
Frequently asked questions
What is a hyperparameter?
A hyperparameter is a setting chosen before training that controls how a model learns — such as the learning rate, batch size, or number of layers — rather than being learned from the data.
What is hyperparameter tuning?
It is the process of searching for the hyperparameter values that give the best performance, often by trying many combinations and evaluating each on validation data.
