#numerical
6 articles
01
·Numerical Computing·★ MEMBER·12 min read
Build Your Own Autograd — A Mini PyTorch in 100 Lines
Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize.
02
·Numerical Computing·FREE·10 min read
How Autodiff Actually Works — Unpacking the PyTorch Magic
Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.
03
·Numerical Computing·★ MEMBER·PAPER·9 min read
The FFT from Scratch — Why Convolution Turns into Multiplication
A from-zero walk through the Fourier transform: a smoothie metaphor, the spinning-needle intuition, the DFT formula, and the divide-and-conquer trick behind the FFT — ending with the polynomial-multiplication view that makes the convolution theorem feel obvious.
04
·Parallel & Distributed·★ MEMBER·10 min read
Why GPUs Are Fast — The Execution Model and the Limits of Parallelism
CPUs and GPUs do not mean the same thing by fast. Where the transistor budget goes, how SIMT bundles 32 threads into a warp, why branch divergence costs you, occupancy and register pressure — and finally Amdahl's law as a way to bound the payoff before you start, plus the profiler counters that tell you when the CPU is the bottleneck.
05
·Numerical Computing·★ MEMBER·8 min read
The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes
Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself.
06
·Numerical Computing·★ MEMBER·8 min read
The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes
Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself.