gradients

Definition

The plural covers the whole collection of per-parameter derivatives that one training step produces. Practitioners talk about gradients flowing back through layers, being accumulated over micro-batches, or being clipped — all framings that treat them as a signal moving through the network rather than as a single number. Vanishing and exploding gradients are the classic failure modes this way of speaking describes.

Explainers using this term

• The Mathematics of Diffusion — Generation Seen Through Scores and SDEsGenerative Modeling by Estimating Gradients of the Data Distribution