compute

Definition

As a noun — the resource and the quantity of it spent, measured in FLOPs or GPU-hours. Scaling-law work made training compute one axis alongside data and parameters; more recent work adds test-time compute, the computation spent while answering through longer chains of thought or repeated sampling. In papers the word is almost always the noun, not the verb.

Explainers using this term

• Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
• Scaling Skepticism — A Genealogy of the "Just Make It Bigger" CritiqueTraining Compute-Optimal Large Language Models
• Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't)Training Compute-Optimal Large Language Models