extrapolation

Definition

Applying a model beyond the range it was fitted on. For language models the specific sense is length extrapolation: does quality hold when the input is longer than anything seen during training? Position encoding largely decides the answer, which is why relative and rotary schemes are compared on exactly this. It is strictly harder than interpolation, where the query falls inside the observed range.