rates

Definition

Almost always learning rates: the step sizes an optimiser takes, and the single hyperparameter most likely to decide whether training works at all. The plural signals either per-parameter adaptivity — each coordinate getting its own rate, as in Adam and Adafactor — or a schedule varying the rate over training. In compression papers the same word means bitrate, quantity per unit, which is unrelated.