What Is Recurrent Neural Network (RNN)?
A Recurrent Neural Network (RNN) is a neural network architecture designed for sequential data that processes inputs one step at a time while maintaining a hidden state that carries information from previous steps, enabling it to model temporal patterns and dependencies.
How Recurrent Neural Network (RNN) Works
RNNs process sequences by looping: at each time step, the network takes the current input and the previous hidden state to produce an output and a new hidden state. This gives the network a form of memory, allowing it to handle variable-length sequences like sentences, time series, or audio. However, standard RNNs struggle with long sequences because gradients can vanish or explode during training, making it hard to capture long-range dependencies. This limitation led to improved variants like LSTMs and GRUs, and eventually to the Transformer architecture, which has largely replaced RNNs for most NLP tasks. RNNs still find use in some time series and real-time processing applications.
Real-World Examples
An early machine translation system using RNNs to process source language sentences word by word before generating translations
A music generation system using an RNN to compose melodies by predicting the next note based on the preceding sequence
A stock price prediction model using an RNN to analyze historical price sequences and forecast future movements