hyper-parameter

Definition

A setting chosen before training begins — learning rate, batch size, depth, regularisation strength — as opposed to the parameters the optimiser learns from data. They matter because a strong architecture with poor hyper-parameters can lose to a weaker one that is well tuned, which is the reason automated search exists. The hyphenated and unhyphenated spellings mean the same thing.

Explainers using this term

• Hyperparameter Search — Hunches, Grids, and Bayesian OptimizationRandom Search for Hyper-Parameter Optimization