high-dimensional

Definition

Describing a space with a very large number of variables — in deep learning, the parameter count itself. High dimension brings the curse of dimensionality: data requirements explode and distance-based intuitions degrade as points become nearly equidistant. It also brings help, which is easy to overlook: some optimization results, such as the relative rarity of bad local minima compared with saddle points, hold precisely because the dimension is large.

Explainers using this term

• Convexity and Optimization — Why Deep Learning Works Even Though It Isn't ConvexIdentifying and attacking the saddle point problem in high-dimensional non-convex optimization