mean

Definition

The arithmetic average, and a building block of variance, RMS and every normalisation layer. Which axis the mean is taken over is exactly what separates BatchNorm from LayerNorm, and whether it is subtracted at all is what separates LayerNorm from RMSNorm. Watch the collision with the ordinary verb 'to mean'; near an equation, assume the statistic.