descent

Definition

Moving parameters in the direction that lowers a loss. Gradient descent and its stochastic minibatch form are the base procedure behind essentially all deep learning; optimizers such as Adam are refinements of how each step is scaled and smoothed. When the objective is something to maximize, such as expected reward in policy gradient methods, the same machinery is described as ascent — the sign flips, the mechanics do not.