rnns

Definition

Recurrent neural networks: models that consume a sequence step by step, carrying a fixed-size hidden state forward. The family includes LSTMs and GRUs and dominated sequence modelling before Transformers. Their trade-off is why they keep resurfacing — constant memory and linear-time inference, but training that parallelises poorly and gradients that fade over long spans. Modern state-space models and linear-attention variants try to keep the inference profile while fixing the training side.