non-convex

Definition

A loss surface with more than one basin, so gradient descent carries no guarantee of reaching the global optimum — unlike the convex case, where every local minimum is global. Deep learning is thoroughly non-convex and works anyway: empirically, most minima that SGD reaches generalize comparably, so the theoretical worry mattered far less than expected. What does matter is which basin you land in and how flat it is.

Explainers using this term

• Convexity and Optimization — Why Deep Learning Works Even Though It Isn't ConvexIdentifying and attacking the saddle point problem in high-dimensional non-convex optimization