#rmsnorm

1 articles

01 ·Deep Learning Basics·★ MEMBER·PAPER·10 min read A History of Normalization Layers — From BatchNorm to RMSNorm The layer that made deep learning actually deep, explained from zero. The internal-covariate-shift controversy behind BatchNorm, why LayerNorm threw away the batch axis, and why every modern LLM converged on RMSNorm.