Students Professional
JA EN

#autodiff

2 articles

01 ·Numerical Computing·★ MEMBER·12 min read Build Your Own Autograd — A Mini PyTorch in 100 Lines Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize. 02 ·Numerical Computing·FREE·10 min read How Autodiff Actually Works — Unpacking the PyTorch Magic Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.