Students Professional
JA EN

#llm

52 articles

01 ·FREE·PAPER·9 min read Chain-of-Thought from Scratch — What Changes When You Make a Model Show Its Work Why does adding "think step by step" make a model solve problems it otherwise fails? A ground-up tour of few-shot CoT, zero-shot CoT and self-consistency — the serialization argument for why it works, the conditions where it doesn't, and how it relates to today's reasoning models. 02 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: One Training Example Keeps On-Policy Distillation Improving for Hundreds of Steps Trained on a single query, on-policy distillation still improves for hundreds of steps and recovers most of full-data OPD's gain. The paper explains this with state coverage and absorption rate, and concludes OPD is data-overfed but algorithm-starved. 03 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: Random Attention — Throwing KV Cache Entries Away at Random Works Just as Well The importance score that decides what leaves the KV cache turns out to buy almost nothing. Keep the prompt, evict uniformly at random inside each head, and you match the strongest prior evictor while serving 32-43% more tokens per second in vLLM. A walkthrough of the measurements across four models and six tasks, and the two controlled experiments that explain them. 04 ·RAG & Retrieval·FREE·8 min read RAG vs Fine-Tuning — Which One, and When The two main ways to make an LLM better, compared on four axes: knowledge freshness, cost, hallucination, and data requirements. From metaphor to math to interactive demos to the mistakes that break production systems. 05 ·RAG & Retrieval·★ MEMBER·PAPER·13 min read Paper walkthrough: Hi-Q — splitting a question down to the granularity your corpus can actually retrieve The real bottleneck in multi-hop QA is a mismatch between the granularity of the question and the granularity of retrievable evidence. Hi-Q answers first, then expands only the nodes that failed — in dependency order. Here is the control rule, the tree, and the measured numbers, from zero background. 06 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper walkthrough: Puro-2B — pretraining a 2B model from scratch for $6.9K on consumer GPUs A team ran 1.4 trillion tokens of pretraining on gaming GPUs and reached Qwen2-1.5B-level quality for roughly $4.4K. Here is the cost structure, the FP8 accounting, the effective learning rate, and the curriculum averaging — from first principles. 07 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: Normalized Low-Rank Adaptation — Why Normalizing LoRA's Entry Matrix Works Rescaling LoRA's down-projection so every column has unit length improves convergence, stability and forgetting resistance at zero extra cost. A ground-up reading of Normalized Low-Rank Adaptation (NoRA) through the lens of a hidden preconditioner. 08 ·Evaluation & Judging·★ MEMBER·PAPER·10 min read Benchmark Contamination — How to Doubt a High Score How to tell whether a benchmark score reflects ability or memorization. Covers the three kinds of contamination, the three detection tools — n-gram overlap, embedding neighbors, membership inference — and where each stops working, plus canary strings and time-based splits, ending in a checklist for reading papers skeptically. 09 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Explained: Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement A third to a half of the teacher's token-level grades are wrong, yet the student improves just as fast either way. This paper traces on-policy distillation's gains not to imitating a teacher but to suppressing the student's own low-probability tokens — and drops the teacher entirely. 10 ·Training & Alignment·★ MEMBER·PAPER·10 min read Paper Walkthrough: PaperGym — Turning One Paper Into a Graded Training Environment for Research Plans A research plan has no answer key, so reinforcement learning has no environment. This paper splits each paper into four drawers and builds the question and the grading criteria from different drawers, producing 20,000 training environments where paraphrasing the prompt no longer earns reward. 11 ·Inference & Serving·★ MEMBER·PAPER·12 min read Paper Walkthrough: DART-SD — Training Tool-Calling Agents Without Flattening the Diamond When a multi-turn tool-calling task has order-independent sub-goals, the set of correct solutions spreads out into a diamond lattice. This walkthrough explains why whole-trajectory imitation crushes that structure, and how locating the first point where a rollout leaves recoverable territory — and supervising only what comes after it — changes the picture. 12 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Walkthrough — J-Zero: Growing the Challenger, the Solver, and the Judge Together from Zero Data A model that writes its own problems, solves them, and grades them — with no external data and no human labels. J-Zero's twist is that the grader learns too, which is what lets it keep improving past the two-iteration wall where prior methods stall. 13 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Agentic Artifact Creation — Where Generation Ends and Construction Begins A survey that reorganizes 259 works around a single unit: the delivered artifact. It defines agentic creation through state, edits, and verification, then works through six artifact families, three evaluation targets, four principles, and six open problems — from first principles. 14 ·Inference & Serving·★ MEMBER·PAPER·11 min read TTPO Explained: Training a Model Mid-Exam, With No Answer Key Majority-vote pseudo-labels are wrong on roughly 85% of competition math prompts. So why does training still work? A ground-up walkthrough of TTPO's asymmetric design: distill the rollouts that agree, penalize the ones that disagree. 15 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: FrontierChallenge — Grading Scientific Work on Whether It Was Actually Delivered A walkthrough of FrontierChallenge, a benchmark that scores whether an agent finishes a scientific workflow. Average scores hit 87.9 while full completion stayed at 20.6% — and 0% in electrochemistry, where the average was 94.9. Worse, 75.5% of failing trajectories still ended by claiming they were done. 16 ·Inference & Serving·★ MEMBER·PAPER·11 min read Structured Output and Constrained Decoding — How to Stop an LLM from Breaking Your JSON Instead of asking a model nicely to return JSON, you can drive the probability of every grammatically illegal token to exactly zero before it is ever sampled. A from-scratch walkthrough of constrained decoding — logit masks, vocabulary indexing, what function calling really does, and why syntax guarantees are not content guarantees. 17 ·Large Language Models·★ MEMBER·PAPER·10 min read Scaling Skepticism — A Genealogy of the "Just Make It Bigger" Critique A fair accounting of the case against "more parameters, more data" — data exhaustion, the reasoning wall, and the world-model dispute. The evidence for scaling gets the same scrutiny as the evidence against it, and every claim comes with the observation that would settle it. 18 ·Inference & Serving·★ MEMBER·10 min read Prompt Caching and Context Design — One Prefix Rule That Moves Your Bill by an Order of Magnitude Are you paying to have the same system prompt re-read on every single request? Prompt caching only works on exact prefixes — and that one rule decides what goes where in your context. Why a single timestamp at the top wipes out everything below it, and how misreading the TTL can make caching 25% more expensive than not caching at all. 19 ·Agents·★ MEMBER·PAPER·10 min read Multi-Agent Design Patterns — Division, Debate, Verification Stack as many agents as you like — if they all fail the same way, you have one agent and a larger bill. The condition under which voting actually helps, written down, then the three patterns that follow from it: division of labour, debate, and adversarial verification — plus when one agent is enough. 20 ·Inference & Serving·FREE·PAPER·10 min read LLM Serving from Scratch — vLLM, Continuous Batching, and Not Letting the GPU Idle Getting a model to run and getting it to handle a hundred users are different problems. Without touching the weights or changing a single output token, the order and grouping of requests can multiply what one GPU delivers. This walks through why — arithmetic intensity, continuous batching, PagedAttention — and why throughput and latency can never both win. 21 ·Large Language Models·FREE·PAPER·9 min read Why Language Models Hallucinate — The Mechanics and What Actually Helps Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation. 22 ·Information Theory·★ MEMBER·PAPER·11 min read Compression Is Prediction Is Intelligence — LLMs Through Information Theory Training a model to guess the next token is training it to shrink a file. Route the prediction through arithmetic coding and "probability model" and "code" turn out to be the same object — which makes cross-entropy loss literally the size of the compressed output. Built up from zero, ending at why the Hutter Prize treats compression ratio as an intelligence test. 23 ·Agents·★ MEMBER·PAPER·8 min read Build Your Own Agent Loop — The Minimal Shape of Tool Calling At the center of every AI agent is a single while loop. We build it from scratch without a framework — the shape of JSON function calls, what ReAct actually left behind, and the stopping conditions where nearly every incident originates. 24 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper walkthrough: Apodex 1.1 — scaling agents around completed work Not a bigger model and not more thinking time — Apodex 1.1 scales two other surfaces: the environments an agent learns in, and the way work is organised across agents. A walkthrough from the task contract to the AgentOS delivery gate, the numbers, and the limits. 25 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps. 26 ·Agents·★ MEMBER·PAPER·11 min read Designing Agent Memory — Short-Term, Long-Term, Episodic An LLM remembers nothing. Conversations only look continuous because you re-send the whole history every turn. Building up from zero: the three-layer split, the function that decides what gets recalled, and the part almost nobody designs — forgetting. 27 ·How Transformers Work·FREE·PAPER·10 min read The Transformer, End to End — One Token's Journey from Embedding to Output Between typing a prompt and getting a word back, a single token gets handed a vector, rewritten in room after room, and finally turned back into language. Tokenizing, embedding, position, attention, feed-forward, residuals, and the output head — walked as one continuous trip rather than a pile of parts. 28 ·Large Language Models·★ MEMBER·PAPER·10 min read Building a Pretraining Corpus — From Web Sludge to Textbook Quality Behind the single line "pretrained on a large corpus of web text" sit four stages: text extraction, quality filtering, deduplication, and mixture weights. This is a from-scratch walkthrough of how a gravel heap called Common Crawl gets sifted into textbook-quality prose — from the MinHash equation to the parameter names you actually touch. 29 ·Security·★ MEMBER·PAPER·9 min read LLM Security — Prompt Injection and How to Actually Defend Against It A language model cannot tell your instructions apart from words printed on a document it was asked to read. This article works from that single fact to direct injection, indirect injection, the tool boundary that decides your blast radius, and the layered defence that actually holds. 30 ·RAG & Retrieval·★ MEMBER·PAPER·10 min read GraphRAG from Scratch — Where Knowledge Graphs Meet Retrieval Rebuild your documents as a web of entities and relationships and you can answer questions ordinary RAG cannot reach — the ones about the corpus as a whole. Extraction, entity resolution, community summarization, and local/global search from first principles, ending with an honest account of when it's overkill. 31 ·Paper Deep-Dives·★ MEMBER·PAPER·13 min read Paper Deep Dive — Large Discovery Models: giving an LLM a value signal for what to try next An LLM can propose candidates but cannot reliably grade its own proposals. Large Discovery Models tilts the LLM's proposal distribution by an acquisition value drawn from a Gaussian process surrogate, so search keeps working when every evaluation is expensive. From the equations to the reported numbers. 32 ·Agents·★ MEMBER·PAPER·11 min read Paper walkthrough: StateM — 95.3% on Terminal-Bench 2.1 and a USD 15 run, without touching a single weight Long-horizon agents fail even when the model underneath can solve every individual step. StateM leaves the weights alone and hardens only the execution system around the agent, reporting 95.3% on Terminal-Bench 2.1 and a final scoring run billed at roughly USD 15 instead of USD 574.68. A ground-up walkthrough of the harness-scaling bet. 33 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough — FACET: Grounding Instruction, Environment, Solution and Verifier in One Executable State A terminal task is a four-part bundle: instruction, environment, reference solution, verifier. FACET builds and runs the environment first, then uses that realized state as shared ground for every other artifact. From 71K agent skills it produces 6,078 validated tasks, and 1.2K successful trajectories lift Qwen3.5 at 4B, 9B and 27B alike. 34 ·Inference & Serving·★ MEMBER·PAPER·15 min read Paper Explained: Agentic ESOpt — Drop Backprop, Jiggle the Weights, and Train Long-Horizon LLM Agents No gradients at all: spawn G perturbed copies of the model, run them in the environment, and nudge the weights toward the ones that scored well. That buys full-parameter updates of a 27B agent at inference-level memory (8.41GB) — and a 12.50-point lead over GRPO on 15-move Sudoku. A ground-up walkthrough of the NUS-led paper. 35 ·Large Language Models·★ MEMBER·PAPER·11 min read Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper. 36 ·Large Language Models·★ MEMBER·PAPER·9 min read Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't) Scaling laws turn 'bigger is smarter' into an equation. This article walks from the Kaplan-era recipe to the Chinchilla paper (Hoffmann et al., 2022) — 400+ training runs, three independent estimation methods, and the 70B-vs-280B showdown — using only the paper itself as evidence. 37 ·Agents·★ MEMBER·PAPER·13 min read Paper Deep-Dive: Recursive Synthesis — Extending Verified Tasks Into 40,000 Long-Horizon Terminal Problems Long-horizon terminal tasks cost hundreds to thousands of dollars each to author. RST mass-produces them for about $0.05 by recursively extending already-verified seeds. Fifteen rounds yielded 37,484 tasks, with oracle solutions growing from a median of 67 lines to 374, and the resulting trajectories genuinely improved Qwen3.5 under SFT and PPO — a walkthrough of the data-synthesis paper from Tencent and collaborators. 38 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: No Gold Answers, No Stronger Teacher — How u-OPSD Distills From Its Own Majority Vote u-OPSD (arXiv:2608.06296) replaces the gold solution in on-policy self-distillation with the model's own majority vote, then corrects only the rollouts that disagree with it. A from-scratch walkthrough grounded solely in the paper. 39 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: Metis — A 'Memory Foundation Model' That Moves Agent Memory Inside the Model Agent memory today is mostly bolted on from the outside via RAG. This paper proposes memory foundation models — models whose forward pass natively stores, forgets, and updates information — and builds Metis, the first prototype. A from-scratch walkthrough of how it works, how well it works, and where it breaks. 40 ·Agents·★ MEMBER·PAPER·10 min read Paper Walkthrough: MerchantBench — Can an LLM Agent Run an Online Store for a Year? Why It Earns Only 27.3% of What Humans Do A close read of MerchantBench, which measures the long-term coherence of LLM agents across a 365-day simulation of running an online store. Why the best configuration reaches only 27.3% of the average human's final net worth, read through three failure patterns: decaying activity, premature withdrawal, and policies frozen against the evidence. 41 ·Agents·★ MEMBER·PAPER·9 min read Paper Walkthrough: Mental World Modeling — A World Model That Advances Minds, Not Just Physics You can track a scene's physics perfectly and still get the next human action wrong. A walkthrough of Mental World Modeling (MWM), which carries beliefs, intentions, emotions and norms as state variables so that every action updates physics and mind together — plus Mentis, the training-free baseline, and its numbers across eight models, straight from the paper. 42 ·Agents·★ MEMBER·PAPER·8 min read Paper Explained: LongHorizon-Harness — Long-Horizon Agent Tasks Are a State-Management Problem, Not an Execution Problem Without touching the model at all, keeping task state outside execution and running a Manage-Execute-Audit loop dramatically improves long-horizon task performance — a walkthrough of Alibaba DreamX team's harness-design paper, from analogy to pseudocode. 43 ·Inference & Serving·★ MEMBER·PAPER·8 min read Speculative Decoding from Scratch — How a Tiny Draft Model Speeds Up an LLM Without Changing a Single Output Speculative decoding lets a small model draft several tokens ahead while a large model verifies them in one batch — provably identical outputs, 2–3x faster in practice. A from-scratch walkthrough of the mechanism and the intuition behind the acceptance rate α, straight from the original paper. 44 ·Model Families·★ MEMBER·PAPER·10 min read The Gemma Family from Scratch — Lineage, Inventions, and Where It Fits Google's open-weight LLM family, walked generation by generation. Distillation, local attention, effective parameters — the invention behind each release — plus the licensing detail that turns into an incident if you miss it, and how to actually run one on your own machine with Ollama. 45 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: DAPD — Breaking the Teacher's "Cheat-Sheet Illusion" in Distillation with Dual Anchors When you distill from a teacher that can see the reference solution, the student learns to act as if invisible answers were still there — the privilege illusion. DAPD (arXiv:2608.01735) traces this failure to information asymmetry and fixes it with two levels of anchoring. A walkthrough grounded strictly in the paper itself. 46 ·Large Language Models·FREE·PAPER·7 min read The Science of Prompt Engineering — What Is Proven and What Is Folklore Does telling a model to think step by step actually work? Working only from the Chain-of-Thought paper (Wei et al., 2022): what the experiments genuinely established — an ability that emerges only above a scale threshold — and where the line falls between that and the incantations everyone repeats. 47 ·Agents·★ MEMBER·PAPER·8 min read LLM Agents from Scratch — Designing the Tool-Use Loop Think, reach for a tool, look at what came back, think again. Working only from the text of the ReAct paper (Yao et al., 2022): why adding language to the action space is the whole idea, what function calling really implements, and the failure modes the authors actually measured — runaway loops and hallucinated tool calls. 48 ·Training & Alignment·★ MEMBER·PAPER·9 min read Instruction Tuning and RLHF from Scratch — How a Model Learns to Follow Orders A pretrained model ignores your instructions not because it lacks the ability but because it was optimised for something else. Working only from the InstructGPT paper (Ouyang et al., 2022): the three stages — SFT, reward model, RL — down to the equations, the claim that a 1.3B model beat a 175B one in human evaluation, and the limits the authors themselves put in writing. 49 ·How Transformers Work·FREE·PAPER·10 min read Positional Encoding from Scratch — From Absolute Positions to RoPE A bare Transformer has no idea what word order is. Starting from why position information is needed at all, this article walks through sinusoidal absolute encodings, learned embeddings, and RoPE — the modern LLM standard — showing exactly why rotation encodes relative position. 50 ·Inference & Serving·FREE·7 min read The KV Cache from Scratch — The Heart of Fast Inference An LLM emits one token at a time. Written naively, every single token costs a full recomputation of the whole sequence — a spectacular waste. The keys and values of past tokens never change again, and that one fact drops an entire order of magnitude. What you pay instead is memory, in an amount you can work out yourself, and that is why batch size and context length hit a ceiling. 51 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Deep Dive — LoRA: Low-Rank Adaptation of Large Language Models: Why Low Rank Is Enough A re-reading of the LoRA paper (Hu et al., 2021) grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open. 52 ·RAG & Retrieval·★ MEMBER·8 min read RAG Fundamentals and Design Patterns — Embeddings, Chunking, Reranking, and Evaluation from Scratch Retrieval-Augmented Generation explained from zero: the core intuition, chunking strategies, hybrid search and reranking, and the evaluation design that matters most.