Deep Learning
Deep Learning is a subset of machine learning that uses multi-layered neural networks to learn complex…
Dropout
Dropout is a regularisation method that randomly ignores some neurons during training to improve generalisation.
Neural Network
Neural Network is a model loosely inspired by the brain, made of layers of connected nodes…
Activation Function
Activation Function is a function applied to a neuron's output that lets neural networks model non-linear…
Softmax
Softmax is a function that turns a set of numbers into probabilities that sum to one,…
Weights
Weights is the learned parameters that determine how strongly inputs influence a neural network's output.
Bias Term
Bias Term is an extra learnable value added in a neuron that shifts the activation, helping…
Transformer
Transformer is a neural network architecture based on attention that powers most modern language and multimodal…
Attention Mechanism
Attention Mechanism is a technique that lets a model weigh which parts of the input matter…
Parameter
Parameter is an internal value a model learns during training; large models have billions of them.
Gradient Descent
Gradient Descent is an optimisation method that iteratively adjusts model parameters to reduce prediction error.
Backpropagation
Backpropagation is the algorithm that calculates how to adjust each parameter in a neural network to…
Loss Function
Loss Function is a measure of how far a model's predictions are from the correct answers,…
Epoch
Epoch is one full pass of the training algorithm over the entire training dataset.
Batch Size
Batch Size is the number of training examples processed before the model updates its parameters.
Learning Rate
Learning Rate is a setting that controls how big each step is when a model updates…
Recurrent Neural Network
Recurrent Neural Network is a neural network designed for sequential data, where each step's output depends…
Overparameterization
Overparameterization is using a model with far more parameters than strictly needed, common in modern deep…