window

Definition

A bounded span of what the model can see at once. The context window is the hard limit on tokens processed together; sliding-window attention is the design where each token attends only to a fixed neighbourhood, trading global reach for linear cost. A larger window is not automatically better use of the input — models often attend unevenly across it, handling the beginning and end more reliably than the middle.

Explainers using this term

• How Long-Context LLMs Work — From RoPE Interpolation to Ring AttentionExtending Context Window of Large Language Models via Positional Interpolation