#implementation
5 articles
01
·Distillation & Compression·★ MEMBER·PAPER·11 min read
Build Your Own Distillation — Growing a Small Model in 100 Lines
The distillation loss fits in twenty lines — and almost everyone who writes it trips on the same three things: the direction of the KL, the choice of reduction, and the missing T². We build the whole rig: freezing the teacher, the loss, the training loop, the teacher-free baseline, a temperature sweep, and four sanity checks that prove the implementation isn't quietly broken.
02
·How Transformers Work·★ MEMBER·PAPER·11 min read
Build Your Own BPE Tokenizer — Learning Merge Rules, and Getting Punished by Japanese
Write the BPE trainer and encoder yourself. Why the artifact of training is an ordered rulebook rather than a vocabulary, how to stop recounting the corpus on every merge, why the first few thousand merge slots in Japanese are spent assembling characters, and how to run a vocabulary-size sweep that actually means something.
03
·How Transformers Work·★ MEMBER·PAPER·12 min read
Build Your Own Mini GPT — A Language Model in 300 Lines
Write a character-level GPT in PyTorch from an empty file: tokenizer, causally masked self-attention, training loop, and temperature sampling — then watch Shakespeare's formatting emerge from nothing but next-character prediction.
04
·Generative Models·★ MEMBER·PAPER·11 min read
Build Your Own Diffusion Model — Starting from MNIST
A diffusion model built up from nothing on 28×28 handwritten digits: the two conditions a noise schedule has to satisfy, how the step number gets injected into a U-Net, and why the sampler adds noise back at the very end — the places you only discover by writing the code yourself.
05
·Agents·★ MEMBER·PAPER·8 min read
Build Your Own Agent Loop — The Minimal Shape of Tool Calling
At the center of every AI agent is a single while loop. We build it from scratch without a framework — the shape of JSON function calls, what ReAct actually left behind, and the stopping conditions where nearly every incident originates.