yarn

Definition

A context-extension method for models using rotary position embeddings. Instead of interpolating every frequency band uniformly, YaRN treats high- and low-frequency components differently and rescales attention temperature, which preserves quality far better than naive position interpolation and needs only a short fine-tuning run. The name is a backronym for Yet another RoPE extensioN — no relation to the JavaScript package manager.