dropout

Definition

A regularizer that randomly zeroes unit activations during training, with a fixed probability per unit. It stops the network from relying on any particular co-adaptation of features and behaves roughly like averaging over an ensemble of thinned subnetworks. Crucially it is disabled at inference, with activations rescaled so expectations match — meaning dropout layers behave differently in train and eval mode, a routine source of bugs when the mode is not switched.