shift

Definition

A systematic change in a distribution between one setting and another — training versus deployment, one hospital versus the next, last year versus this year. It is the standard explanation for models that score well offline and degrade in production, and unlike noise it does not average out. The word almost always appears compounded: distribution shift, covariate shift, label shift.

Explainers using this term

• A History of Normalization Layers — From BatchNorm to RMSNormBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift