length

Definition

Almost always sequence length — how many tokens a model can take in. It matters because standard attention costs grow quadratically with it, so context length is bought with compute and memory, which is what motivates sparse, linear, and cached attention variants. A separate question, evaluated separately, is whether a model still works beyond the length it was trained on.

Explainers using this term

• Paper Explained: When EOS Tokens Disagree — Why On-Policy Distillation Students Forget How to StopWhen EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation