hyperparameter

Definition

The knobs set before training — learning rate, number of layers, weight decay — which gradient descent does not touch. Because they are not learned, they must be found by search or by hand, and tuning effort becomes a hidden variable in most comparisons. A fair benchmark gives every baseline a comparable tuning budget; papers that do not say so invite doubt about the margin they report.