#pytorch
5 articles
01
·Distillation & Compression·★ MEMBER·PAPER·11 min read
Build Your Own Distillation — Growing a Small Model in 100 Lines
The distillation loss fits in twenty lines — and almost everyone who writes it trips on the same three things: the direction of the KL, the choice of reduction, and the missing T². We build the whole rig: freezing the teacher, the loss, the training loop, the teacher-free baseline, a temperature sweep, and four sanity checks that prove the implementation isn't quietly broken.
02
·How Transformers Work·★ MEMBER·PAPER·12 min read
Build Your Own Mini GPT — A Language Model in 300 Lines
Write a character-level GPT in PyTorch from an empty file: tokenizer, causally masked self-attention, training loop, and temperature sampling — then watch Shakespeare's formatting emerge from nothing but next-character prediction.
03
·Generative Models·★ MEMBER·PAPER·11 min read
Build Your Own Diffusion Model — Starting from MNIST
A diffusion model built up from nothing on 28×28 handwritten digits: the two conditions a noise schedule has to satisfy, how the step number gets injected into a U-Net, and why the sampler adds noise back at the very end — the places you only discover by writing the code yourself.
04
·Linear Algebra·FREE·10 min read
Tensors and Shape Manipulation — If You Can Read einsum, You Can Read Papers
The Σ_j A_ij B_jk in the paper and the x.transpose(1,2) in the code say the same thing, and einsum is the bridge between them. Three tools — axes, broadcasting, contraction — are enough to write attention in a single line.
05
·Numerical Computing·FREE·10 min read
How Autodiff Actually Works — Unpacking the PyTorch Magic
Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.