covariate

Definition

The input-side variables of a model — the statistician's term for predictors, carried into ML unchanged. Covariate shift names a specific failure mode: the distribution of inputs changes between training and deployment while the conditional distribution of the label given the input stays fixed. That specificity matters, since it is the assumption that makes importance weighting valid; label shift needs different treatment.

Explainers using this term

• A History of Normalization Layers — From BatchNorm to RMSNormBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift