#nlp
11 articles
01
·How Transformers Work·★ MEMBER·PAPER·11 min read
Build Your Own BPE Tokenizer — Learning Merge Rules, and Getting Punished by Japanese
Write the BPE trainer and encoder yourself. Why the artifact of training is an ordered rulebook rather than a vocabulary, how to stop recounting the corpus on every merge, why the first few thousand merge slots in Japanese are spent assembling characters, and how to run a vocabulary-size sweep that actually means something.
02
·How Transformers Work·★ MEMBER·PAPER·11 min read
Build Your Own BPE Tokenizer — Learning Merge Rules, and Getting Punished by Japanese
Write the BPE trainer and encoder yourself. Why the artifact of training is an ordered rulebook rather than a vocabulary, how to stop recounting the corpus on every merge, why the first few thousand merge slots in Japanese are spent assembling characters, and how to run a vocabulary-size sweep that actually means something.
03
·How Transformers Work·★ MEMBER·PAPER·12 min read
Build Your Own Mini GPT — A Language Model in 300 Lines
Write a character-level GPT in PyTorch from an empty file: tokenizer, causally masked self-attention, training loop, and temperature sampling — then watch Shakespeare's formatting emerge from nothing but next-character prediction.
04
·How Transformers Work·★ MEMBER·PAPER·12 min read
Build Your Own Mini GPT — A Language Model in 300 Lines
Write a character-level GPT in PyTorch from an empty file: tokenizer, causally masked self-attention, training loop, and temperature sampling — then watch Shakespeare's formatting emerge from nothing but next-character prediction.
05
·How Transformers Work·★ MEMBER·PAPER·9 min read
Encoder or Decoder — The Fork in the Road Between BERT and GPT
One masked triangle in the attention table is what separated BERT from GPT. This piece works through bidirectional versus autoregressive with the equations and an interactive figure, then asks why generation won and where encoders are still the first choice.
06
·How Transformers Work·★ MEMBER·PAPER·9 min read
Encoder or Decoder — The Fork in the Road Between BERT and GPT
One masked triangle in the attention table is what separated BERT from GPT. This piece works through bidirectional versus autoregressive with the equations and an interactive figure, then asks why generation won and where encoders are still the first choice.
07
·How Transformers Work·FREE·PAPER·10 min read
Tokenizers from Scratch — The Unit an LLM Cuts the World Into
An LLM reads neither characters nor words. How BPE builds a vocabulary, what SentencePiece actually fixed, why some languages pay more for the same sentence, and what you trade away when you grow the vocabulary — worked by hand and in code, from zero.
08
·How Transformers Work·FREE·PAPER·10 min read
Tokenizers from Scratch — The Unit an LLM Cuts the World Into
An LLM reads neither characters nor words. How BPE builds a vocabulary, what SentencePiece actually fixed, why some languages pay more for the same sentence, and what you trade away when you grow the vocabulary — worked by hand and in code, from zero.
09
·RAG & Retrieval·FREE·PAPER·6 min read
Embeddings from Scratch — from word2vec Intuition to Contextual Embeddings
Why does 'king − man + woman ≈ queen' actually work? A from-zero tour of embeddings: the map metaphor, the distributional hypothesis, the word2vec math, numpy code, contextual embeddings after BERT, and the knobs you touch in real RAG systems.
10
·Paper Deep-Dives·★ MEMBER·PAPER·12 min read
Paper Deep Dive — Attention Is All You Need: What Dropping Recurrence Actually Proved
A close reading of the Transformer paper grounded strictly in its own text: the scaled dot-product equation, why the square root of d_k is there, what the ablations exposed, and the limits the authors themselves flagged.
11
·How Transformers Work·FREE·8 min read
Attention from Scratch — The Heart of the Transformer, Explained Visually
Self-attention, the core mechanism behind ChatGPT, explained from zero: analogy, intuition, matrix mechanics, and runnable numpy code. No math background required.