recurrent

Definition

Processing a sequence step by step while carrying a hidden state forward, as in RNNs, LSTMs, and GRUs. The appeal is constant memory regardless of sequence length; the drawback that cost them the field is poor parallelism during training, which is where Transformers won. Recurrence has returned in modern state-space models, which recover linear-time inference while remaining trainable in parallel.

Explainers using this term

• Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27BWhy Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
• Diagnosing Broken Training — Telling Divergence, NaN, and Plateaus ApartOn the difficulty of training Recurrent Neural Networks
• Paper explained: BDH-CQ — an AI that thinks without words. Recurrent memory plus latent reasoning resets ARC's cost frontierBDH-CQ: In-Context Learning with Recurrent Latent Reasoning