Numerical Computing
GEMM, decompositions, FFT, iterative methods — where AI's compute actually goes
01
·Numerical Computing·★ MEMBER·8 min read
The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes
Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself.
02
·Numerical Computing·★ MEMBER·PAPER·9 min read
The FFT from Scratch — Why Convolution Turns into Multiplication
A from-zero walk through the Fourier transform: a smoothie metaphor, the spinning-needle intuition, the DFT formula, and the divide-and-conquer trick behind the FFT — ending with the polynomial-multiplication view that makes the convolution theorem feel obvious.
03
·Numerical Computing·★ MEMBER·10 min read
Numerical Pitfalls — Cancellation, Rounding, and logsumexp
Where "the loss went nan three hours into the run" actually comes from, built up from nothing: how rounding enters, how the condition number amplifies it, and why subtracting two close numbers is so destructive. It all converges on logsumexp — the one trick sitting inside every softmax and cross-entropy implementation.
04
·Numerical Computing·FREE·10 min read
How Autodiff Actually Works — Unpacking the PyTorch Magic
Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.
05
·Numerical Computing·★ MEMBER·13 min read
Solving Systems of Equations — Direct Methods and Iterative Methods
How much a bridge sags, how heat spreads through a room, what a Gaussian process predicts — once a computer gets hold of them they all turn into the same shape, Ax = b. This article builds up elimination (LU) and approximation (conjugate gradients) from zero, through why a million-unknown system can't be solved by elimination, all the way to condition numbers, preconditioning, and matrix-free solvers.
06
·Numerical Computing·★ MEMBER·12 min read
Build Your Own Autograd — A Mini PyTorch in 100 Lines
Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize.