Recurrent Neural Network (RNN) is a type of neural network designed to handle sequences — like text or time series — by processing data step by step while retaining a memory of what came before.
What it means in plain English
Ordinary networks treat each input independently, which is a poor fit for sequences where order matters. RNNs address this by feeding their output from one step back into the next, giving them a form of memory. This let them handle tasks like language and speech, where understanding depends on context from earlier in the sequence. However, RNNs struggled to remember information across long sequences.
That limitation is a key reason transformers, with their attention mechanism, largely replaced RNNs for language tasks.
A simple example
An early predictive-text system might use an RNN: as you type, it processes each word in order, keeping a memory of the sentence so far to guess the next word — though it could lose track over long passages.
Why it matters
RNNs were an important step in teaching AI to handle sequential data like language, and understanding them clarifies the problem that transformers solved. They remain part of the story of how modern language AI developed.
Related terms
- Transformer — the architecture that largely replaced RNNs for language.
- Attention Mechanism — the innovation that overcame RNN limitations.
- Neural Network — the broader family.
Frequently asked questions
What are RNNs used for?
Recurrent neural networks process sequences — like text, speech, or time series — by maintaining a memory of previous inputs, making them suited to sequential data.
Have transformers replaced RNNs?
For most language tasks, transformers have largely replaced RNNs due to better handling of long-range context and parallel training, though RNNs still appear in some sequence applications.
