interpolation

Definition

Estimating values between known ones. In long-context work it names Position Interpolation: rather than asking a model to extrapolate to position indices it never saw, you rescale the indices so long inputs fold back into the trained range, after which brief fine-tuning restores quality. The contrast with extrapolation is the whole point — interpolating inside a familiar range is far more reliable than pushing beyond it. The word also describes blending between two points in a latent space.

Explainers using this term

• How Long-Context LLMs Work — From RoPE Interpolation to Ring AttentionExtending Context Window of Large Language Models via Positional Interpolation