accelerating

Definition

Reaching the same result in less time or fewer steps — the classic use is Batch Normalization: Accelerating Deep Network Training. Be careful about what is actually accelerated: wall-clock time on fixed hardware, number of optimisation steps to a target loss, and throughput from better parallelism are different claims covered by one word. The body, not the title, tells you which.

Explainers using this term

• A History of Normalization Layers — From BatchNorm to RMSNormBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift