compute-matched

Definition

States that the systems being compared were given the same compute budget, so a score difference reflects the method rather than the spend. Without it, almost any improvement is confounded — more training FLOPs, more sampling at inference, a longer schedule. Note this is not the same as parameter-matched: two models of identical size can differ by an order of magnitude in training tokens or in forward passes per answer.

Explainers using this term

• Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers