Learn › AI

Large Language Models

Scaling laws, prompting, and how LLMs behave inside

01 ·Large Language Models·FREE·PAPER·9 min read Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't) Scaling laws turn 'bigger is smarter' into an equation. This article walks from the Kaplan-era recipe to the Chinchilla paper (Hoffmann et al., 2022) — 400+ training runs, three independent estimation methods, and the 70B-vs-280B showdown — using only the paper itself as evidence. 02 ·Large Language Models·FREE·PAPER·7 min read The Science of Prompt Engineering — What Is Proven and What Is Folklore Does telling a model to think step by step actually work? Working only from the Chain-of-Thought paper (Wei et al., 2022): what the experiments genuinely established — an ability that emerges only above a scale threshold — and where the line falls between that and the incantations everyone repeats. 03 ·Large Language Models·FREE·PAPER·12 min read Mamba and State Space Models — Handling Sequences Without Attention Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention. 04 ·Large Language Models·FREE·PAPER·10 min read Building a Pretraining Corpus — From Web Sludge to Textbook Quality Behind the single line "pretrained on a large corpus of web text" sit four stages: text extraction, quality filtering, deduplication, and mixture weights. This is a from-scratch walkthrough of how a gravel heap called Common Crawl gets sifted into textbook-quality prose — from the MinHash equation to the parameter names you actually touch. 05 ·Large Language Models·FREE·PAPER·11 min read LLM Evaluation from Scratch — Reading Benchmarks and the Contamination Problem A guide to reading the bar charts in model release posts with the right kind of suspicion. Covers how the scoring method alone moves MMLU numbers, the error bar that comes from question count, why public benchmarks get contaminated structurally rather than accidentally, the Bradley-Terry model behind Chatbot Arena and where it breaks, and the three biases in LLM-as-a-judge. 06 ·Large Language Models·FREE·PAPER·9 min read Why Language Models Hallucinate — The Mechanics and What Actually Helps Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation. 07 ·Large Language Models·FREE·PAPER·13 min read Test-Time Scaling — How Models Get Better by Thinking Longer The same model scores higher when you let it think longer. This article builds the idea from scratch: chain-of-thought as purchased compute steps, self-consistency by majority vote, verifiers that pick the winner, and o1-style models that learned the thinking itself — and what it means for compute to shift from training to inference. 08 ·Large Language Models·FREE·PAPER·10 min read Knowledge Distillation from Scratch — Copying a Big Model into a Small One No label ever says "dogs look a bit like cats." A trained model's output does — and that gap is what knowledge distillation harvests. Soft labels, temperature-scaled softmax, where the T² factor comes from, sequence-level and synthetic-data distillation, and how DeepSeek-R1 copied an entire reasoning procedure into smaller models. 09 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps. 10 ·Large Language Models·★ MEMBER·PAPER·11 min read Scaling Skepticism — A Genealogy of the "Just Make It Bigger" Critique A fair accounting of the case against "more parameters, more data" — data exhaustion, the reasoning wall, and the world-model dispute. The evidence for scaling gets the same scrutiny as the evidence against it, and every claim comes with the observation that would settle it. 11 ·Large Language Models·★ MEMBER·PAPER·8 min read Paper Walkthrough: Can Anything Catch a Fake Crisis Video? — What RA-Bench Found Sixteen thousand AI videos, each continuing from the real first frame of a genuine disaster or war clip, put against seven classical detectors, ten zero-shot multimodal models and two purpose-built fine-tunes. None of them generalized. One model turned out to be reading timestamps rather than pixels, and a lap through a social feed drops fake recall to 1.4%. 12 ·Large Language Models·★ MEMBER·PAPER·11 min read Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper. 13 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Agentic Artifact Creation — Where Generation Ends and Construction Begins A survey that reorganizes 259 works around a single unit: the delivered artifact. It defines agentic creation through state, edits, and verification, then works through six artifact families, three evaluation targets, four principles, and six open problems — from first principles. 14 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper walkthrough: Puro-2B — pretraining a 2B model from scratch for $6.9K on consumer GPUs A team ran 1.4 trillion tokens of pretraining on gaming GPUs and reached Qwen2-1.5B-level quality for roughly $4.4K. Here is the cost structure, the FP8 accounting, the effective learning rate, and the curriculum averaging — from first principles. 15 ·Large Language Models·★ MEMBER·PAPER·12 min read Paper walkthrough: RealSWE — re-measuring coding agents when the request is just "fix it" A paper that measures, on real data, how far SWE-bench problem statements sit from actual user requests, then builds 381 task families where only the information content and writing style change. Resolution rates fall 6.4 pp on average — and the ranking flips. 16 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper Explained: FlowBalance — Verifier-Grounded Self-Improvement from a Model's Own Reasoning Learning from your own answers breaks in two ways: the grading is too sparse, or you believe your own confidence. FlowBalance fixes both with one move — let the verifier decide the sign of the dense self-score — and learns a normalized distribution over whole responses. Explained from first principles. 17 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper Walkthrough — Feyospace-v1: Seven People, 164k Trajectories, and a Cyber Model A seven-person team pushed open-weight models to leading agentic cyber capability — not by scaling, but by building resettable executable environments and auditing 164,269 trajectories. A ground-up walkthrough of the design and its stated limits. 18 ·Large Language Models·★ MEMBER·PAPER·7 min read Paper Explained: OmniEdu — Building an Open Education LLM Around Four Skills: Solve, Place, Diagnose, Guide A from-scratch walkthrough of OmniEdu, an open K-12 LLM family: how ~1.34M candidate examples were curated down to 60,951, how per-example teaching instructions let one model either answer or hold back, and what the results and limitations actually show. 19 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Beyond Solver Verdicts — When the Solver Says Yes but the Translation Is Wrong An LLM translates a problem into logic and Z3 solves it. This paper names the failure where the solver's verdict is right but the translation is wrong (VPU), and distills Z3's equivalence check into a Yes/No next-token verifier called GenV. Explained from zero. 20 ·Large Language Models·★ MEMBER·PAPER·8 min read Paper Explained: Training Object Permanence in World Models — Teaching Video Models That Hidden Things Still Exist Video generators make hidden objects vanish and let balls pass through walls. We walk through WROP, a 150-task Blender benchmark for object permanence and solidity, the 16B PWM-WROP model fine-tuned on it, and how to read the human-preference evaluation behind the results. 21 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Just-in-Time Memory — Curate Agent Memory When You Read It, Not When You Write It A from-scratch walkthrough of JitMem, which stores raw agent trajectories and only summarizes them once the next task is known. Why that makes the curator easy to train, and exactly how far the reported numbers go. 22 ·Large Language Models·★ MEMBER·PAPER·8 min read Paper Explained: Bellman Policy Optimization — Policy Mirror Descent for LLMs, Without a Value Model In LLM reinforcement learning where only the final answer is scored, BPO (arXiv:2609.15987) targets the same optimum as Policy Mirror Descent without estimating a single intermediate state value. We walk from the Bellman 'telescoping sum' to the practical loss, using only the paper itself as the source.