Large Language Models
Scaling laws, prompting, and how LLMs behave inside
01
·Large Language Models·★ MEMBER·PAPER·9 min read
Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't)
Scaling laws turn 'bigger is smarter' into an equation. This article walks from the Kaplan-era recipe to the Chinchilla paper (Hoffmann et al., 2022) — 400+ training runs, three independent estimation methods, and the 70B-vs-280B showdown — using only the paper itself as evidence.
02
·Large Language Models·FREE·PAPER·7 min read
The Science of Prompt Engineering — What Is Proven and What Is Folklore
Does telling a model to think step by step actually work? Working only from the Chain-of-Thought paper (Wei et al., 2022): what the experiments genuinely established — an ability that emerges only above a scale threshold — and where the line falls between that and the incantations everyone repeats.
03
·Large Language Models·FREE·PAPER·12 min read
Mamba and State Space Models — Handling Sequences Without Attention
Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention.
04
·Large Language Models·★ MEMBER·PAPER·10 min read
Building a Pretraining Corpus — From Web Sludge to Textbook Quality
Behind the single line "pretrained on a large corpus of web text" sit four stages: text extraction, quality filtering, deduplication, and mixture weights. This is a from-scratch walkthrough of how a gravel heap called Common Crawl gets sifted into textbook-quality prose — from the MinHash equation to the parameter names you actually touch.
05
·Large Language Models·★ MEMBER·PAPER·11 min read
LLM Evaluation from Scratch — Reading Benchmarks and the Contamination Problem
A guide to reading the bar charts in model release posts with the right kind of suspicion. Covers how the scoring method alone moves MMLU numbers, the error bar that comes from question count, why public benchmarks get contaminated structurally rather than accidentally, the Bradley-Terry model behind Chatbot Arena and where it breaks, and the three biases in LLM-as-a-judge.
06
·Large Language Models·FREE·PAPER·9 min read
Why Language Models Hallucinate — The Mechanics and What Actually Helps
Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation.
07
·Large Language Models·★ MEMBER·PAPER·13 min read
Test-Time Scaling — How Models Get Better by Thinking Longer
The same model scores higher when you let it think longer. This article builds the idea from scratch: chain-of-thought as purchased compute steps, self-consistency by majority vote, verifiers that pick the winner, and o1-style models that learned the thinking itself — and what it means for compute to shift from training to inference.
08
·Large Language Models·★ MEMBER·PAPER·9 min read
Knowledge Distillation from Scratch — Copying a Big Model into a Small One
No label ever says "dogs look a bit like cats." A trained model's output does — and that gap is what knowledge distillation harvests. Soft labels, temperature-scaled softmax, where the T² factor comes from, sequence-level and synthetic-data distillation, and how DeepSeek-R1 copied an entire reasoning procedure into smaller models.
09
·Large Language Models·★ MEMBER·PAPER·8 min read
Alignment, Explained — From RLHF to Constitutional AI
What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps.
10
·Large Language Models·★ MEMBER·PAPER·10 min read
Scaling Skepticism — A Genealogy of the "Just Make It Bigger" Critique
A fair accounting of the case against "more parameters, more data" — data exhaustion, the reasoning wall, and the world-model dispute. The evidence for scaling gets the same scrutiny as the evidence against it, and every claim comes with the observation that would settle it.
11
·Large Language Models·★ MEMBER·PAPER·8 min read
Paper Walkthrough: Can Anything Catch a Fake Crisis Video? — What RA-Bench Found
Sixteen thousand AI videos, each continuing from the real first frame of a genuine disaster or war clip, put against seven classical detectors, ten zero-shot multimodal models and two purpose-built fine-tunes. None of them generalized. One model turned out to be reading timestamps rather than pixels, and a lap through a social feed drops fake recall to 1.4%.
12
·Large Language Models·★ MEMBER·PAPER·11 min read
Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced
When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper.
13
·Large Language Models·★ MEMBER·PAPER·9 min read
Paper Explained: Agentic Artifact Creation — Where Generation Ends and Construction Begins
A survey that reorganizes 259 works around a single unit: the delivered artifact. It defines agentic creation through state, edits, and verification, then works through six artifact families, three evaluation targets, four principles, and six open problems — from first principles.
14
·Large Language Models·★ MEMBER·PAPER·13 min read
Paper walkthrough: Puro-2B — pretraining a 2B model from scratch for $6.9K on consumer GPUs
A team ran 1.4 trillion tokens of pretraining on gaming GPUs and reached Qwen2-1.5B-level quality for roughly $4.4K. Here is the cost structure, the FP8 accounting, the effective learning rate, and the curriculum averaging — from first principles.