looped

Definition

Describes an architecture that feeds activations through the same block repeatedly instead of through a stack of distinct layers. Because the weights are shared, extra iterations buy effective depth and extra inference compute without adding parameters, which makes looped models a natural way to spend more compute at test time on harder inputs. Contrast with ordinary depth, where each layer holds its own parameters and cannot be re-run for extra thinking.

Explainers using this term

• Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers