#numerical

7 articles

01 ·Numerical Computing·FREE·PAPER·16 min read The Fourier Transform from Scratch — Reading a Signal as a Stack of Pure Tones A signal drawn against time hides the thing you usually want to know: which frequencies are in it, and how much of each. This article builds the Fourier transform from one idea — measure a signal against a pure tone with an inner product — then takes the continuous integral apart symbol by symbol, does the same for the DFT, separates two tones by hand, and ends on the three traps that bite in practice: aliasing, finite resolution, and spectral leakage. 02 ·Numerical Computing·★ MEMBER·13 min read Build Your Own Autograd — A Mini PyTorch in 100 Lines Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize. 03 ·Numerical Computing·FREE·10 min read How Autodiff Actually Works — Unpacking the PyTorch Magic Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine. 04 ·Numerical Computing·★ MEMBER·PAPER·9 min read The FFT from Scratch — Why Convolution Turns into Multiplication A from-zero walk through the Fourier transform: a smoothie metaphor, the spinning-needle intuition, the DFT formula, and the divide-and-conquer trick behind the FFT — ending with the polynomial-multiplication view that makes the convolution theorem feel obvious. 05 ·Parallel & Distributed·FREE·10 min read Why GPUs Are Fast — The Execution Model and the Limits of Parallelism CPUs and GPUs do not mean the same thing by fast. Where the transistor budget goes, how SIMT bundles 32 threads into a warp, why branch divergence costs you, occupancy and register pressure — and finally Amdahl's law as a way to bound the payoff before you start, plus the profiler counters that tell you when the CPU is the bottleneck. 06 ·Numerical Computing·FREE·8 min read The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself. 07 ·Numerical Computing·FREE·8 min read The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself.