long-context

Definition

The ability to take tens of thousands to millions of tokens in a single forward pass, and the research area around it. It matters because a whole codebase, contract, or conversation history can be handed over without chunking, which is either a replacement for or a complement to retrieval. The standing caveat: accepting a long input and actually using all of it are different capabilities, and benchmarks exist to separate them.

Explainers using this term

• Paper Walkthrough: Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training — training the draft alongside a 122B policy at 256K tokensOnline Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training