internal

Definition

Occurring inside the model rather than at its interface with the world. The key compound is internal covariate shift: as earlier layers update, the distribution of activations feeding later layers keeps moving, so each layer chases a target its own predecessors keep displacing. Distinguish it from ordinary covariate shift, where the change originates in the data or the deployment environment.

Explainers using this term

• A History of Normalization Layers — From BatchNorm to RMSNormBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift