#gradient-descent

2 articles

01 ·Calculus & Optimization·FREE·PAPER·10 min read Gradient Descent from Scratch — The Geometry of Learning Every model that learns is walking downhill on a surface it cannot see all of. This article builds gradient descent from that one picture — the loss surface, the gradient as steepest ascent, the update rule w ← w − η∇L — then works out why the learning rate has a hard ceiling, why ravines make it zigzag, and what local minima really cost in high dimensions. 02 ·Machine Learning Basics·FREE·8 min read Loss Functions and Optimization — How a Model Learns From Being Wrong Why MSE and cross-entropy have the shapes they do, what the gradient actually points at, and one step of gradient descent taken apart with equations, a draggable figure, and ten lines of numpy — divergence included.