#backpropagation
4 articles
01
·Numerical Computing·★ MEMBER·12 min read
Build Your Own Autograd — A Mini PyTorch in 100 Lines
Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize.
02
·Calculus & Optimization·★ MEMBER·11 min read
Matrix Calculus from Scratch — Derive the Backward Pass Yourself
Where does the transpose in ∂L/∂W = XᵀG actually come from? Matrix calculus is not a formula sheet to memorize — it is one move: rotate dX to the right inside a trace. From denominator layout and shape-checking to the gradients of a linear layer and softmax + cross-entropy, ending with a double-precision gradient check.
03
·Numerical Computing·FREE·10 min read
How Autodiff Actually Works — Unpacking the PyTorch Magic
Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.
04
·Deep Learning Basics·★ MEMBER·9 min read
Backpropagation from Scratch — It Is All Just the Chain Rule
Why you can get gradients for ten million parameters for roughly the cost of one forward pass. The chain rule, computational graphs, a two-layer network worked by hand with real numbers, and where vanishing gradients come from — every symbol explained as it appears.