stochastic

Definition

Involving deliberate randomness. The canonical case is stochastic gradient descent, which estimates the gradient from a random minibatch instead of the full dataset — cheaper per step, and the injected noise turns out to help generalisation as well. The contrast is with deterministic procedures, which return the same result on every run given the same input.

Explainers using this term

• Beyond SGD — Adam, Second-Order Methods, and Constrained OptimizationAdam: A Method for Stochastic Optimization