Students Professional
JA EN

#text

295 articles

01 ·FREE·PAPER·9 min read Chain-of-Thought from Scratch — What Changes When You Make a Model Show Its Work Why does adding "think step by step" make a model solve problems it otherwise fails? A ground-up tour of few-shot CoT, zero-shot CoT and self-consistency — the serialization argument for why it works, the conditions where it doesn't, and how it relates to today's reasoning models. 02 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: Terminal-Universe — Turning Agent Logs Back Into Reusable Execution Environments Replay the file operations recorded in an agent trajectory, have a completion agent fill in what's missing, and you get an executable workspace back. The pipeline yields 37.3k environments and lifts Qwen3.5-27B by 11.9 points on Terminal-Bench 2.1. 03 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: One Training Example Keeps On-Policy Distillation Improving for Hundreds of Steps Trained on a single query, on-policy distillation still improves for hundreds of steps and recovers most of full-data OPD's gain. The paper explains this with state coverage and absorption rate, and concludes OPD is data-overfed but algorithm-starved. 04 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: Random Attention — Throwing KV Cache Entries Away at Random Works Just as Well The importance score that decides what leaves the KV cache turns out to buy almost nothing. Keep the prompt, evict uniformly at random inside each head, and you match the strongest prior evictor while serving 32-43% more tokens per second in vLLM. A walkthrough of the measurements across four models and six tasks, and the two controlled experiments that explain them. 05 ·Large Language Models·FREE·PAPER·12 min read Mamba and State Space Models — Handling Sequences Without Attention Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention. 06 ·Agents·★ MEMBER·PAPER·9 min read CogEvol: What the Reward Cannot Measure, RL Will Quietly Destroy A technical report on a model family that generates teaching material in a single pass. Its centerpiece is an incident the authors disclose in full: a screenshot-only reward taught the policy to ship games that looked convincing and could not be played. 07 ·How Transformers Work·★ MEMBER·PAPER·10 min read Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27B A walkthrough of the paper that tested — and overturned — the belief that the recurrent half of a hybrid LLM is too fragile for 4-bit quantization, and explained mechanistically why log-space gates and the delta rule erase quantization noise. 08 ·How Transformers Work·★ MEMBER·PAPER·10 min read Paper Walkthrough: Stop Anchoring to Frame One — Scal3R's Multi-Reference Relative Pose Query Long videos make online 3D reconstruction collapse. This paper traces the cause to extrapolation against a fixed first-frame anchor and fixes it by adding roughly 1% of trainable tokens to a frozen backbone. Explained from zero. 09 ·RAG & Retrieval·FREE·8 min read RAG vs Fine-Tuning — Which One, and When The two main ways to make an LLM better, compared on four axes: knowledge freshness, cost, hallucination, and data requirements. From metaphor to math to interactive demos to the mistakes that break production systems. 10 ·Inference & Serving·★ MEMBER·PAPER·9 min read Paper Walkthrough: Language Models Can Control Their Own Attention During long-context decoding, a model re-reads its entire KV cache at every step. Declarative Attention has the model announce, inside its own chain-of-thought, where it will look next, and the inference engine builds an attention mask from that text. A walkthrough of the paper that cut attended tokens by 52.0% and 31.1% on off-the-shelf models, zero-shot. 11 ·RAG & Retrieval·★ MEMBER·PAPER·13 min read Paper walkthrough: Hi-Q — splitting a question down to the granularity your corpus can actually retrieve The real bottleneck in multi-hop QA is a mismatch between the granularity of the question and the granularity of retrievable evidence. Hi-Q answers first, then expands only the nodes that failed — in dependency order. Here is the control rule, the tree, and the measured numbers, from zero background. 12 ·Inference & Serving·★ MEMBER·PAPER·10 min read Paper Explained: Compile by Training — Turning a Natural-Language Spec into a Function That Runs Locally "Sort my email into urgent and later" — a spec that vague, turned into a small function you can run locally after about a minute of training. A ground-up walkthrough of Compile by Training: analogy, mechanism, equations, measured numbers. 13 ·Training & Alignment·★ MEMBER·PAPER·13 min read Paper Walkthrough: It Takes Two to Match — Co-Evolving Both Sides of Retrieval with RL A paper that rebuilds the first stage of search by having two LLMs write keywords — one for queries, one for items — and training them in alternation until their vocabularies meet. Explained from scratch. 14 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: HarnessDev — Can an LLM Build and Maintain the System It Runs Inside? A ground-up walkthrough of HarnessDev, a benchmark that asks whether an LLM can build an agent harness from a deliberately useless seed and then improve it from execution feedback. Models can build one; the gains rarely survive contact with held-out tasks or a different runtime model. 15 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: EarlyEval — Making Agent Evaluation Cheaper by Stopping Early One evaluation pass over an agentic benchmark costs hundreds of dollars. EarlyEval exploits the fact that an agent's ending is legible from its middle, halting runs to cut 13–26% of steps. Mechanism, numbers, and limits, straight from the paper. 16 ·Agents·★ MEMBER·PAPER·14 min read Paper Walkthrough: Aspire — Can Models Self-Evolve from Vague Goals? Hand an agent nothing but "get better at mathematical reasoning" and let it decide what to learn and how to check itself. Aspire measures the result on 520 hidden, expert-written items — and finds that agents close the training loop far more reliably than the capability loop. 17 ·Agents·★ MEMBER·PAPER·11 min read Paper Explained: What Makes Good Agentic Data? The ACE Lens A survey that recasts agentic training data as a four-part object (environment, task, interaction, verifier) and reframes generation as constrained distribution design: admit on Accuracy, place mass by Complexity, spread coverage with divErsity. 18 ·Agents·★ MEMBER·PAPER·15 min read Paper Walkthrough: Training Agents to Evolve with Their Harness Why compact models break when you rewrite prompts and tool definitions every week, and what the team behind Taobao Live's AI avatar streamers does about it. A ground-up walkthrough of Harness-Aware Training, from intuition to equations to measured numbers and stated limits. 19 ·Training & Alignment·★ MEMBER·PAPER·12 min read Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model. 20 ·Agents·★ MEMBER·PAPER·13 min read Paper Explained: StarHarness — Evolving the Scaffold Instead of the Weights Freeze the model weights and search over the harness instead — prompts, tool schemas, skills, MCP providers, subagents, agent loop. Across three enterprise benchmarks this bought 20–35 points, held on tasks never used for evolution, and transferred to other models unchanged. 21 ·Inference & Serving·★ MEMBER·PAPER·15 min read Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched? A study of looped Transformers that finally controls the comparison: per-token FLOPs, total parameters, and KV cache are all held close. The resulting recipe, SMELT, loops the middle half twice and reports 6.8–18.0% training-FLOPs savings on the compute-optimal frontier. 22 ·Evaluation & Judging·★ MEMBER·PAPER·12 min read Self-Improving AI — Self-Play, Co-Evolution, and Generated Curricula What has to be true for a model to get better without anyone adding data? This article pulls three conditions out of AlphaZero's self-play, shows exactly which one breaks first for language models, explains how co-evolution and generated curricula try to patch the gap, and ends with why self-improvement claims are unusually easy to evaluate wrong. 23 ·Evaluation & Judging·★ MEMBER·PAPER·10 min read Reward Hacking — Whatever You Measure Is Where It Breaks The moment you pick a metric, that metric starts to rot. This piece explains why Goodhart's law is statistically unavoidable, walks through real failures from boat races that spin in circles to RLHF verbosity, sycophancy and hardcoded unit tests, and covers how to detect the gap between optimization pressure and true performance. 24 ·Agents·★ MEMBER·PAPER·12 min read Paper Explained: Repo-To-Skill — Distilling GitHub Repositories Into Skills an AI Can Use A third layer that is neither the model nor the harness: operational knowledge, distilled from 1,000 GitHub repositories into 5,353 verified skills. What DisCo and the AREX-Skill Library actually do, what 31.11%→72.89% on MLE-bench is made of, and where the skills failed. 25 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper walkthrough: Puro-2B — pretraining a 2B model from scratch for $6.9K on consumer GPUs A team ran 1.4 trillion tokens of pretraining on gaming GPUs and reached Qwen2-1.5B-level quality for roughly $4.4K. Here is the cost structure, the FP8 accounting, the effective learning rate, and the curriculum averaging — from first principles. 26 ·Agents·★ MEMBER·PAPER·15 min read Paper Walkthrough: PILOT in the Loop — Fixing the Run While It Is Still Running Agent self-improvement that only starts after execution ends arrives too late. A ground-up walkthrough of PILOT, a supervisor–worker harness that redirects the live run while distilling skills from it, from intuition to equations to measured numbers and stated limits. 27 ·How Transformers Work·★ MEMBER·PAPER·16 min read Paper Walkthrough: Designing Qwen3.8-Next — Accuracy, Efficiency and Stability as One Problem A ground-up read of the Qwen3.8-Flash-Next design report: the GDN hybrid, Qwen Sparse Attention, the Gated Residual and n-gram embeddings, judged the way the paper judges them — on loss, on cost, and on stability. 28 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: Normalized Low-Rank Adaptation — Why Normalizing LoRA's Entry Matrix Works Rescaling LoRA's down-projection so every column has unit length improves convergence, stability and forgetting resistance at zero extra cost. A ground-up reading of Normalized Low-Rank Adaptation (NoRA) through the lens of a hidden preconditioner. 29 ·Agents·★ MEMBER·PAPER·11 min read Paper Walkthrough: LoopArena — Benchmarking the Model That Steers a Coding Agent A ground-up walkthrough of LoopArena, a benchmark that scores the outer loop steering a coding agent while holding the agent itself fixed — including how to read the headline result of 24.69% strict success on full tasks. 30 ·Evaluation & Judging·FREE·PAPER·10 min read LLM-as-a-Judge from Scratch — How AI Grades AI, and Where It Breaks A ground-up guide to using one model to grade another. Covers reading a verdict as a probability distribution, the three recurring biases (position, verbosity, self-enhancement), why pairwise comparison cost grows quadratically, and how to validate the judge itself against human labels. 31 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: From Production Traffic to Post-Training — Folding 200 Internal Apps Into One Self-Hosted LLM How a company forced to self-host by data-residency rules collapsed a sprawling model fleet into one checkpoint: classify production failures by hand, train one GRPO expert per weak axis, then merge the weights with SLERP. Includes three real reward-hacking failures. 32 ·Evaluation & Judging·★ MEMBER·PAPER·10 min read Benchmark Contamination — How to Doubt a High Score How to tell whether a benchmark score reflects ability or memorization. Covers the three kinds of contamination, the three detection tools — n-gram overlap, embedding neighbors, membership inference — and where each stops working, plus canary strings and time-based splits, ending in a checklist for reading papers skeptically. 33 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Explained: Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement A third to a half of the teacher's token-level grades are wrong, yet the student improves just as fast either way. This paper traces on-policy distillation's gains not to imitating a teacher but to suppressing the student's own low-probability tokens — and drops the teacher entirely. 34 ·Agents·★ MEMBER·PAPER·11 min read Paper Walkthrough: AutoSaddler — Growing a Harness That Doesn't Break, from Agent Failure Logs A ground-up walkthrough of AutoSaddler, which automatically optimizes the harness around an LLM agent — prompts, tools, and middleware — by repeatedly diagnosing failure traces and generating structured patches. It beat the base harnesses on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 by 9.0, 9.6, and 10.0 points. 35 ·Evaluation & Judging·FREE·PAPER·11 min read A Map of Agent Benchmarks — What SWE-bench, GAIA, and OSWorld Actually Measure What the three most-cited agent benchmarks — SWE-bench, GAIA, and OSWorld — really measure, read off from how each one grades. Includes the four distinct kinds of contamination and a checklist for translating a published score into something useful for your own system. 36 ·Training & Alignment·★ MEMBER·PAPER·10 min read Paper Walkthrough: PaperGym — Turning One Paper Into a Graded Training Environment for Research Plans A research plan has no answer key, so reinforcement learning has no environment. This paper splits each paper into four drawers and builds the question and the grading criteria from different drawers, producing 20,000 training environments where paraphrasing the prompt no longer earns reward. 37 ·Inference & Serving·★ MEMBER·PAPER·12 min read Paper Walkthrough: DART-SD — Training Tool-Calling Agents Without Flattening the Diamond When a multi-turn tool-calling task has order-independent sub-goals, the set of correct solutions spreads out into a diamond lattice. This walkthrough explains why whole-trajectory imitation crushes that structure, and how locating the first point where a rollout leaves recoverable territory — and supervising only what comes after it — changes the picture. 38 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Walkthrough — J-Zero: Growing the Challenger, the Solver, and the Judge Together from Zero Data A model that writes its own problems, solves them, and grades them — with no external data and no human labels. J-Zero's twist is that the grader learns too, which is what lets it keep improving past the two-iteration wall where prior methods stall. 39 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper walkthrough: CyberFactory — turning wild CVEs into runnable training problems An open-source pipeline that converts real CVEs into executable, verifiable tasks and uses a reusable vulnerability-analysis skill to synthesize teacher trajectories the student then internalizes. 58.1% Pass@1 on CyberGym. 40 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Agentic Artifact Creation — Where Generation Ends and Construction Begins A survey that reorganizes 259 works around a single unit: the delivered artifact. It defines agentic creation through state, edits, and verification, then works through six artifact families, three evaluation targets, four principles, and six open problems — from first principles. 41 ·Inference & Serving·★ MEMBER·PAPER·11 min read TTPO Explained: Training a Model Mid-Exam, With No Answer Key Majority-vote pseudo-labels are wrong on roughly 85% of competition math prompts. So why does training still work? A ground-up walkthrough of TTPO's asymmetric design: distill the rollouts that agree, penalize the ones that disagree. 42 ·Inference & Serving·★ MEMBER·PAPER·13 min read Paper explained: Self-OPD — an image generator that distills itself, with no teacher No teacher model at all: at every denoising step the student spawns K copies of itself, scores them, and pulls toward the good branches while pushing away from the bad ones. A ground-up walkthrough of Self-OPD, an alignment method for flow matching image generators. 43 ·Paper Deep-Dives·FREE·13 min read When Proxies Stop Being Good Enough — Reading August 2026's Eight Autonomous Driving Papers Together A cross-cutting read of eight autonomous-driving arXiv papers from late August 2026. Three groups independently stop measuring safety in expectation, two add an observation channel outside the ego vehicle's own history, and two genuinely don't fit the story. 44 ·Agents·★ MEMBER·PAPER·13 min read Paper Explained: JIT-Agent — A Model That Writes the Agent Harness On Demand An agent's capability is not the model's alone. This is a ground-up walkthrough of JIT-Agent, a model trained to synthesize the four-module harness — memory, planning, action, tools — freshly for each task. 45 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough — WarpSAC: When RL's Safety Rails Become Handcuffs Once GPU-parallel simulators flood the replay buffer, SAC's normalization and clipped double-Q flip from helpful to restrictive. The paper isolates three design axes and prescribes removing stabilizers — not stacking them — when data are abundant. 46 ·Engineering Process·FREE·8 min read Upstream Engineering from Scratch — Why Projects Are Won or Lost at Requirements Requirements, then high-level design, then detailed design — why that order? The reason is a branching factor that multiplies at every stage, making the cost of a late fix grow exponentially. Explained from zero with an analogy, a formula, an interactive plot, and code. 47 ·Supply Chain·FREE·8 min read Mapping the Semiconductor Supply Chain — From Sand to Chip, Who Holds What One pass through the whole chain that makes a chip — design (EDA/IP), equipment, materials, foundry, OSAT — and why nearly every stage collapses to a handful of firms, read through two formulas: chain availability and HHI. 48 ·Supply Chain·★ MEMBER·13 min read Upstream of Semiconductors — Wafers, Photoresist, and Specialty Gases How sand becomes silicon that is 99.999999999% pure, gets pulled out of a melt as one giant single crystal, and is coated with a film that dissolves where light hits it. From segregation in the Czochralski process to acid blur in chemically amplified resist — and why a handful of Japanese firms still own this layer. 49 ·Supply Chain·★ MEMBER·9 min read Inside the Equipment Makers — What ASML, AMAT, TEL and Lam Actually Build Chip manufacturing is four actions repeated hundreds of times — deposit, print, etch, clean — and each action has its own machine and its own specialist vendor. After mapping steps to suppliers, two pieces of arithmetic explain why the tool business collapses into a handful of firms, and why ASML's EUV monopoly is really a bundle of irreplaceable component makers. 50 ·Engineering Process·★ MEMBER·12 min read The Craft of Requirements — Why "We Built Exactly What They Asked For" Fails Why shipping precisely what a stakeholder asked for produces software that works and nobody uses. Stated versus latent needs, use cases, turning "fast" and "secure" into numbers, and a set of question templates that expose ambiguity on the spot. 51 ·Distillation & Compression·★ MEMBER·PAPER·9 min read On-Policy Distillation — Learning From What the Student Actually Writes Classic distillation has the student copy sentences the teacher wrote. On-policy distillation has the student write, then lets the teacher mark it up. The difference is one symbol in the loss — and that symbol removes exposure bias, turns distillation into a form of RL, and opens the door to self-distillation methods like u-OPSD and AgentOPSD. 52 ·Supply Chain·★ MEMBER·11 min read IP Cores and the Fabless Model — How Arm Rules Silicon Without Making a Single Chip Why a company that only sells blueprints ends up at the center of the chip industry. License fees versus royalties, the shipment volume where buying beats building, the three grades of IP (soft, hard, architectural), and what RISC-V actually made free — and what it did not. 53 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: FrontierChallenge — Grading Scientific Work on Whether It Was Actually Delivered A walkthrough of FrontierChallenge, a benchmark that scores whether an agent finishes a scientific workflow. Average scores hit 87.9 while full completion stayed at 20.6% — and 0% in electrochemistry, where the average was 94.9. Worse, 75.5% of failing trajectories still ended by claiming they were done. 54 ·Engineering Process·★ MEMBER·10 min read Estimation and Scope — The Cone of Uncertainty and How to Negotiate Estimates miss because a question about a range gets answered with a point. The cone of uncertainty, three-point estimates, why buffers don't add up the way you think, reference class forecasting, and how to turn a deadline conversation into a scope conversation. 55 ·Supply Chain·★ MEMBER·10 min read EDA Tools from Scratch — Chips Are Written in Software Nobody hand-places the tens of billions of transistors on a leading-edge chip. Designers write text (RTL), and software compiles it into physical geometry through synthesis, place-and-route, and verification. Here is that pipeline from zero, plus why Synopsys and Cadence became impossible to route around. 56 ·Distillation & Compression·★ MEMBER·PAPER·10 min read Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller Model There are three roads to a smaller model: coarsen the number grid (quantization), remove weights outright (pruning), or rebuild the thing at a smaller size (distillation). A head-to-head comparison on compression ratio, accuracy, and implementation cost — and why, when you stack them, training-based methods go first and quantization goes last. 57 ·Distillation & Compression·★ MEMBER·PAPER·9 min read A Field Guide to Distillation Recipes — logit, feature, attention, self Every distillation recipe is an answer to one question: which part of the teacher should the student match? This guide lines up output (logit), intermediate feature (FitNet), attention, and self-distillation in one table and four equations, then turns the choice into a decision you make from constraints — is the teacher behind an API, do the layers line up, do the head counts match. 58 ·Distillation & Compression·FREE·PAPER·8 min read The Math of Distillation — Why Soft Answers Teach More Why is the distillation loss KL(teacher||student), what is the temperature T actually doing, and where does that mysterious T² factor in every implementation come from? A ground-up derivation of why a soft distribution carries more teaching signal than a correct answer. 59 ·Distillation & Compression·★ MEMBER·PAPER·10 min read Distilling Agents — How to Compress a Long Trajectory When you distill an agent, the unit of learning changes from a single question-and-answer to an entire game. Tens of thousands of tokens come back with one bit of grading. This article works through how that gap is bridged — turn-level credit assignment, trajectory filtering and on-policy correction, and the inheritance of tool use — and maps the recent long-horizon agent literature onto those three axes. 60 ·Distillation & Compression·★ MEMBER·PAPER·8 min read When Distillation Fails — Capacity Gaps and Contagious Overconfidence A stronger teacher does not guarantee a stronger student. A student too small to hold the teacher's structure returns a blurred average; a confident teacher hands down its errors along with its confidence; and the student walks paths the teacher never traveled. The three ways distillation breaks, from first principles. 61 ·Distillation & Compression·★ MEMBER·PAPER·9 min read Evaluating Distilled Models — Is "Close to the Teacher" a Good Metric? Score a distilled student by how often it agrees with its teacher and the students that faithfully reproduce the teacher's mistakes come out on top. What agreement actually guarantees, what breaks outside the training distribution, and how to test for contamination that arrives paraphrased through synthetic data. 62 ·Distillation & Compression·★ MEMBER·PAPER·8 min read Designing Distillation Data — Deciding What to Ask the Teacher What decides a distilled student's quality is less how smart the teacher is than what you asked the teacher to answer. Synthetic data generation, coverage design, why you should skew toward hard problems, correctness filtering, and why DeepSeek-R1's distillation worked — from first principles. 63 ·Supply Chain·★ MEMBER·9 min read The Geopolitics of Chips — Export Controls and Supply Chain Rewiring, Explained Technically Why chokepoints form exactly where they do, from three conditions: physics, fixed cost, and tacit knowledge. Covers the technical reason controls are written as numeric thresholds, the common-cause formula that kills dual sourcing, and the learning curve that sets how fast a substitute can arrive. 64 ·Distillation & Compression·★ MEMBER·PAPER·11 min read Build Your Own Distillation — Growing a Small Model in 100 Lines The distillation loss fits in twenty lines — and almost everyone who writes it trips on the same three things: the direction of the KL, the choice of reduction, and the missing T². We build the whole rig: freezing the teacher, the loss, the training loop, the teacher-free baseline, a temperature sweep, and four sanity checks that prove the implementation isn't quietly broken. 65 ·Engineering Process·★ MEMBER·9 min read Architecture Decisions — Telling Apart What You Can Undo From What You Cannot How to separate the design decisions you can cheaply reverse from the ones you can never take back. Measuring reversibility, writing ADRs that record what you gave up, and a template for putting trade-offs into words — from first principles. 66 ·Engineering Process·★ MEMBER·9 min read Upstream Work in the AI Era — When Code Gets Cheap, What Gets Valuable? When implementation gets cheap, which parts of the cost remain, and what becomes expensive by comparison? We put the relationship between deciding and reworking into an equation, then design specs for AI as lossy compression of intent — starting from zero assumed knowledge. 67 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read Video Understanding from Scratch — From a Pile of Frames to a Sense of Time A video is not just a lot of images. This piece splits video understanding into three questions — which frames to take (sampling), how to mix them (temporal attention), and how to shrink a long video (compression) — and works through each with equations, interactive figures, and code. 68 ·Training & Alignment·FREE·PAPER·11 min read Diagnosing Broken Training — Telling Divergence, NaN, and Plateaus Apart Training breaks in exactly three ways: it diverges, it goes NaN, or it stalls. Built around a symptom-to-cause table, this article shows why divergence is a threshold effect (with the math and an interactive figure), how to pin down where a NaN was born, and how to isolate the cause of a plateau — assuming no prior knowledge. 69 ·Generative Models·★ MEMBER·PAPER·10 min read The State of 3D Generation — From NeRF to Gaussian Splatting 3D stopped being something you sculpt and became something you fit by optimization. How the representation moved from meshes to NeRF to Gaussian Splatting, how SDS turns a 2D diffusion model into a judge that supervises 3D, and what breaks when you take any of it into a game or film pipeline. 70 ·Large Language Models·★ MEMBER·PAPER·13 min read Test-Time Scaling — How Models Get Better by Thinking Longer The same model scores higher when you let it think longer. This article builds the idea from scratch: chain-of-thought as purchased compute steps, self-consistency by majority vote, verifiers that pick the winner, and o1-style models that learned the thinking itself — and what it means for compute to shift from training to inference. 71 ·Paper Deep-Dives·★ MEMBER·11 min read Symbolic vs. Connectionist — Where a 60-Year Argument Stands Today AI has always had two tribes: one that writes the rules down, one that learns weights from examples. Tracing the Perceptrons book, the collapse of expert systems, and the backpropagation comeback explains why today's LLMs still call calculators and get shackled to grammars. 72 ·Agents·★ MEMBER·PAPER·9 min read Paper Walkthrough: SWE-bench Science — Can Coding Agents Fix Scientific Code? A walkthrough of SWE-bench Science, a repository-level benchmark of 119 tasks from 98 repos across 20 scientific domains. The best agent scores under 50% pass@1, and the paper names four recurring failure mechanisms plus an ablation showing scientific knowledge is not uniformly helpful. 73 ·Inference & Serving·★ MEMBER·PAPER·11 min read Structured Output and Constrained Decoding — How to Stop an LLM from Breaking Your JSON Instead of asking a model nicely to return JSON, you can drive the probability of every grammatically illegal token to exactly zero before it is ever sampled. A from-scratch walkthrough of constrained decoding — logit masks, vocabulary indexing, what function calling really does, and why syntax guarantees are not content guarantees. 74 ·Probability & Statistics·★ MEMBER·PAPER·11 min read Statistical Learning Theory — Why Does Learning Generalize? You only ever see finitely many examples, yet the model answers questions about data it has never seen. This piece builds the guarantee from scratch — Hoeffding, uniform convergence, VC dimension, PAC — then shows where it broke against deep learning, and what survives in day-to-day evaluation design. 75 ·Machine Learning Basics·★ MEMBER·PAPER·11 min read Self-Supervised Learning — The Day Unlabeled Data Became an Asset Nobody has to label the data — the data can write its own exam. A ground-up tour of the two families (masked prediction and contrastive learning), from the intuition through the InfoNCE loss, an interactive figure, PyTorch code, and finally why LLM pretraining is the largest self-supervised system ever run. 76 ·Generative Models·★ MEMBER·PAPER·11 min read The Mathematics of Diffusion — Generation Seen Through Scores and SDEs A layer beneath 'add noise, then subtract it': diffusion models restated in the language of the score, the gradient of log-density. Why denoising and score estimation are literally the same job, what the forward SDE, reverse SDE and probability flow ODE actually assert, and how those equations turn into the knobs you set at inference time. 77 ·Large Language Models·★ MEMBER·PAPER·10 min read Scaling Skepticism — A Genealogy of the "Just Make It Bigger" Critique A fair accounting of the case against "more parameters, more data" — data exhaustion, the reasoning wall, and the world-model dispute. The evidence for scaling gets the same scrutiny as the evidence against it, and every claim comes with the observation that would settle it. 78 ·RAG & Retrieval·★ MEMBER·PAPER·13 min read Evaluating RAG in Practice — Turning “Seems Better” Into a Number RAG improvement work gets lost because quality is judged as one vague feeling. This article builds up the metrics that separate retrieval from generation, the definitions of faithfulness and relevance, how to build an eval set with synthetic QA and where that goes wrong, and how many questions you need before the number means anything. 79 ·Inference & Serving·★ MEMBER·10 min read Prompt Caching and Context Design — One Prefix Rule That Moves Your Bill by an Order of Magnitude Are you paying to have the same system prompt re-read on every single request? Prompt caching only works on exact prefixes — and that one rule decides what goes where in your context. Why a single timestamp at the top wipes out everything below it, and how misreading the TTL can make caching 25% more expensive than not caching at all. 80 ·Probability & Statistics·★ MEMBER·PAPER·10 min read Optimal Transport — The Mathematics of Moving Distributions Optimal transport measures the distance between two probability distributions as the cheapest way to haul sand from one pile into the shape of another. From the definition of the Wasserstein distance to the Sinkhorn algorithm that makes it practical, and on to WGAN, FID and Flow Matching — with numpy code and the failure modes that bite in production. 81 ·Model Families·★ MEMBER·11 min read Open vs Closed — The Economics of Releasing Weights, and the Safety Argument Why would a company give away weights that cost a fortune to train? The decisive difference between open-weight and open-source, the licence clauses that actually cause incidents, and what each side of the release debate is really claiming — plus the convergence nobody expected. From no prior knowledge. 82 ·Device Physics·★ MEMBER·12 min read The Physics of NAND Flash — Remembering by Trapping Electrons Why does a memory stick keep your photos with the power off? Starting from one idea — trap electrons on an island of insulator and shift a threshold voltage — this article builds up tunnelling writes, multi-level cells, 3D stacking and wear-out, and ends where you can read an SSD datasheet and a SMART log for yourself. 83 ·Agents·★ MEMBER·PAPER·10 min read Multi-Agent Design Patterns — Division, Debate, Verification Stack as many agents as you like — if they all fail the same way, you have one agent and a larger bill. The condition under which voting actually helps, written down, then the three patterns that follow from it: division of labour, debate, and adversarial verification — plus when one agent is enough. 84 ·Probability & Statistics·★ MEMBER·10 min read Monte Carlo Methods from Scratch — Solving Integrals with Dice An integral you cannot solve can still be estimated: scatter random points and take the average. This piece builds up why the law of large numbers buys you only 1/√N, what importance sampling rescues, and how MCMC samples from a distribution whose normalizing constant nobody can compute. 85 ·Machine Learning Basics·★ MEMBER·8 min read ML System Design — The 90% Outside the Model The accuracy you hit in a notebook is not a promise about production. Feature definitions, training-serving skew, monitoring that catches slow decay, and the retraining loop — the 90% that lives outside the model, laid out in the order you actually design it. 86 ·Model Families·★ MEMBER·9 min read The Mistral Family from Scratch — Europe's Small-and-Strong Bet How a Paris startup made 'small but strong' work: the sliding-window attention behind Mistral 7B, the sparse mixture of experts behind Mixtral, the Codestral code line, and the licence policy that started at Apache 2.0 and then wandered — explained from no prior knowledge. 87 ·Agents·FREE·11 min read MCP and Tool Protocols — The Standard That Connects an Agent's Hands When an LLM touches your calendar or your database, what is actually wired to what? From what a tool call really is, to the N×M problem MCP solves, to designing tool definitions, to the security boundary you cannot design around — starting from zero. 88 ·How Transformers Work·★ MEMBER·PAPER·11 min read How Long-Context LLMs Work — From RoPE Interpolation to Ring Attention A "128K context window" is two different walls, knocked down by two unrelated families of tricks. This walks through positional interpolation, NTK-aware scaling and YaRN for the position wall; sliding windows and ring attention for the compute wall; and how to read a needle-in-a-haystack chart without being fooled by it. 89 ·Inference & Serving·FREE·PAPER·10 min read LLM Serving from Scratch — vLLM, Continuous Batching, and Not Letting the GPU Idle Getting a model to run and getting it to handle a hundred users are different problems. Without touching the weights or changing a single output token, the order and grouping of requests can multiply what one GPU delivers. This walks through why — arithmetic intensity, continuous batching, PagedAttention — and why throughput and latency can never both win. 90 ·Large Language Models·★ MEMBER·PAPER·11 min read LLM Evaluation from Scratch — Reading Benchmarks and the Contamination Problem A guide to reading the bar charts in model release posts with the right kind of suspicion. Covers how the scoring method alone moves MMLU numbers, the error bar that comes from question count, why public benchmarks get contaminated structurally rather than accidentally, the Bradley-Terry model behind Chatbot Arena and where it breaks, and the three biases in LLM-as-a-judge. 91 ·Coding Theory·★ MEMBER·11 min read LDPC and Turbo Codes — The Error Correction Behind 5G and Deep Space Shannon proved in 1948 that these codes exist, and nobody could build one for 45 years. Starting from zero background, this article unpacks the three ideas — soft decisions, log-likelihood ratios, and belief propagation — that finally reached the theoretical limit, and shows what actually runs in 5G NR and deep-space links. 92 ·Large Language Models·★ MEMBER·PAPER·9 min read Knowledge Distillation from Scratch — Copying a Big Model into a Small One No label ever says "dogs look a bit like cats." A trained model's output does — and that gap is what knowledge distillation harvests. Soft labels, temperature-scaled softmax, where the T² factor comes from, sequence-level and synthetic-data distillation, and how DeepSeek-R1 copied an entire reasoning procedure into smaller models. 93 ·Probability & Statistics·★ MEMBER·PAPER·11 min read Kernel Methods and Gaussian Processes — The Champions Before Neural Nets Before deep learning took over, kernel SVMs held the crown in classification. This article builds the whole story from zero: the kernel trick that makes lifting into high dimensions free, Gaussian processes that put a probability distribution over functions, and Bayesian optimization that turns uncertainty into a weapon. 94 ·Calculus & Optimization·★ MEMBER·11 min read Jacobians and Hessians — Multivariable Calculus, Drawn The Jacobian is a magnifying glass at a point; the Hessian is how sharply the ground curves there. From local linearization to what eigenvalues say about the terrain, why Newton's method is fast on paper but absent in practice, and how to get Hessian information in a billion dimensions without ever building the matrix. 95 ·Numerical Computing·★ MEMBER·13 min read Solving Systems of Equations — Direct Methods and Iterative Methods How much a bridge sags, how heat spreads through a room, what a Gaussian process predicts — once a computer gets hold of them they all turn into the same shape, Ax = b. This article builds up elimination (LU) and approximation (conjugate gradients) from zero, through why a million-unknown system can't be solved by elimination, all the way to condition numbers, preconditioning, and matrix-free solvers. 96 ·Computer Architecture·★ MEMBER·9 min read Interconnects — How NVLink, PCIe, and Light Set the Limits of Scale When adding GPUs stops making things faster, the wiring is usually to blame. This piece pins down the bandwidth hierarchy from HBM through NVLink, PCIe, and the inter-node fabric, breaks collective time into 'steps × latency + bytes ÷ bandwidth', and explains why all-to-all hurts and why distance eventually demands light. 97 ·Inference & Serving·★ MEMBER·11 min read Cutting Inference Cost in Practice — What to Do First Quantization and distillation both work, but in most shops there are two or three levers left that cost you nothing in quality. Break the bill into four numbers, then reorder the work: cache, batch, shorten, compress. What each lever actually buys, and what breaks when you take them out of order. 98 ·Machine Learning Basics·★ MEMBER·10 min read Imbalanced Data in Practice — What to Optimize When 99% Is Normal When only 1% of your data is positive, accuracy lies. We build up from the confusion matrix to PR curves, class weighting, resampling, probability calibration, and deriving the decision threshold from cost — no prior knowledge assumed. 99 ·CNNs & Image Recognition·FREE·PAPER·9 min read The ImageNet Moment — The Day Deep Learning Won In 2012 an image-recognition contest saw its error rate fall from around 26% to 15% in a single year. Neural networks had existed for three decades — so why then? This is the story of the moment data, compute, and method finally lined up, told down to the technical details. 100 ·Generative Models·FREE·PAPER·11 min read A Practical Map of Image Generation — SD, ControlNet, and Applying LoRA A single map for anyone about to touch image generation for the first time: the four boxes of latent diffusion, where the prompt actually takes effect, locking down composition with ControlNet, fine-tuning with LoRA, and the licensing layers that quietly cause the most damage. 101 ·Deep Learning Basics·★ MEMBER·PAPER·12 min read Hyperparameter Search — Hunches, Grids, and Bayesian Optimization Gradients tell you nothing about the learning rate, so you have to go looking. Why grid search is weak, why search spaces should be carved on a log scale, what a Bayesian acquisition function is actually counting, and why early stopping beats a cleverer search algorithm — with Optuna code and the traps that bite in production. 102 ·Large Language Models·FREE·PAPER·9 min read Why Language Models Hallucinate — The Mechanics and What Actually Helps Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation. 103 ·Inference & Serving·★ MEMBER·PAPER·11 min read Surviving GPU Out-of-Memory — Every Cause, Every Fix `CUDA out of memory` reports only the allocation that happened to fail last, which is almost never the culprit. We count what actually occupies VRAM in five buckets, derive the sixteen-bytes-per-parameter fixed cost of training, and work through the fixes in order of least damage: gradient checkpointing, optimizer compression, offloading, KV cache limits, and fragmentation. 104 ·Deep Learning Basics·★ MEMBER·PAPER·12 min read Graph Neural Networks from Scratch — Learning from Connections Social networks, molecules and road maps are all built from dots and lines. This piece builds message passing up from nothing, explains why stacking layers makes every node look identical (over-smoothing), and follows the idea through to how recommenders and drug discovery actually use it. 105 ·Agents·★ MEMBER·PAPER·13 min read Paper Explained: FreeToken — Treating Your Own PC as a Single Elastic Inference Platform FreeToken is an edge-native serving system for frontier-scale MoE models on personal hardware. Its centerpiece is a q* policy that decides how many missed experts to ship over PCIe versus execute in place on the CPU — using nothing but two measured bandwidths. 106 ·How Transformers Work·★ MEMBER·PAPER·10 min read FlashAttention from Scratch — The Paradox of Doing More Math to Go Faster FlashAttention deliberately recomputes the same math in the backward pass. It does strictly more arithmetic and still runs faster — because on a GPU, moving numbers costs more than multiplying them. From counting HBM round trips, through the softmax wall that blocks tiling, to the online-softmax recurrence that breaks it, and the silent backend fallback that bites people in production. 107 ·OS & Runtime·★ MEMBER·9 min read File Systems from Scratch — What Does "Saved" Actually Guarantee? If the power dies half a second after your editor says "Saved," is the file still there? Starting from a library card catalogue, this builds up blocks, inodes, directories, journaling and fsync from zero — and ends with why databases refuse to trust the file system, and which knobs turn a quiet setting into a production incident. 108 ·Linear Algebra·★ MEMBER·PAPER·10 min read Symmetry and Equivariance — How Group Theory Shapes Network Design How to bake "a shifted cat is still a cat" into the architecture itself — from the four axioms of a group, through the definition of equivariance, to why convolution is the only translation-equivariant linear map and how AlphaFold handles 3D geometry. Including the cases where imposing symmetry backfires. 109 ·Training & Alignment·★ MEMBER·PAPER·13 min read DPO and What Came After — The Lineage That Simplified RLHF Derives DPO one line at a time, starting from the closed-form solution to KL-constrained reward maximization, to show why no separate reward model is needed. Then organizes IPO (which explains DPO's overfitting mathematically), KTO (which drops the pairing requirement), and GRPO (which drops the value model and goes back online) by what each one deleted — and gives a rule for choosing based on the shape of the data you actually have. 110 ·Databases·★ MEMBER·PAPER·10 min read Distributed Databases — CAP, Replication and Consensus The moment you add a second copy of your database, a brand-new question appears: which copy is right? Replication, quorums, CAP, two-phase commit, Raft and eventual consistency — from zero assumed knowledge down to the parameters you actually set in PostgreSQL and Cassandra. 111 ·Training & Alignment·★ MEMBER·10 min read Versioning Data and Models — An Experiment You Cannot Reproduce Never Happened If you cannot get "92% accuracy" back six months later, it was an anecdote, not an experiment. We build up content-addressing (naming things by their contents), lineage (the graph from inputs to outputs), and the three levels of "how much is worth pinning down" — from the hash math to a manifest you can write today to the mistakes that quietly ruin a year of runs. 112 ·Machine Learning Basics·★ MEMBER·10 min read Data Leakage and Experiment Hygiene — When the Score Is Too Good, Suspect It When a model scores better than you expected, that is not good news yet. This article splits data leakage into five concrete shapes — outcome-derived columns, preprocessing before the split, time running backwards, duplicates and groups, and a worn-out test set — then covers the detection moves that actually find them and the experiment records that let you dissect the day the number jumped. 113 ·Parallel & Distributed·★ MEMBER·9 min read Concurrency from Scratch — Locks, Atomics, and Memory Models Why data races happen and why they refuse to reproduce in your tests, starting from zero. Locks, atomic operations, CAS, and memory models, built up through metaphor, math, and code. 114 ·Information Theory·★ MEMBER·PAPER·11 min read Compression Is Prediction Is Intelligence — LLMs Through Information Theory Training a model to guess the next token is training it to shrink a file. Route the prediction through arithmetic coding and "probability model" and "code" turn out to be the same object — which makes cross-entropy loss literally the size of the compressed output. Built up from zero, ending at why the Hutter Prize treats compression ratio as an intelligence test. 115 ·Machine Learning Basics·★ MEMBER·PAPER·11 min read Unsupervised Learning from Scratch — Clustering and Dimensionality Reduction How to pull structure out of data that carries no labels at all, assuming nothing. What k-means, hierarchical clustering and DBSCAN are each good for, what PCA is actually maximizing, and the three ways people misread a t-SNE or UMAP picture — metaphor, then formula, then a figure you can move, then code. 116 ·Training & Alignment·★ MEMBER·PAPER·9 min read Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep Learning Why a model that learns something new tends to lose what it already knew, explained from a single fact: weights are a shared resource. Walks through EWC, replay, and swappable LoRA adapters with equations and interactive figures, then explains why production teams still just mix the old data back in and retrain. 117 ·Calculus & Optimization·★ MEMBER·13 min read Calculus of Variations — What It Means to Differentiate a Function When the thing you are searching for is a shape rather than a number, differentiation has to be reinvented. Functionals and the first variation, the Euler–Lagrange equation, why the fastest slide is a cycloid, and how all of it lands on the ELBO — the variational problem machine learning runs most. 118 ·RAG & Retrieval·★ MEMBER·PAPER·13 min read Build Your Own Vector DB — From Brute Force to HNSW Assemble a vector search engine step by step, starting from a 20-line brute-force scan. The curse of dimensionality, IVF partitioning, HNSW graph traversal and quantization, all viewed through one lens: the trade between recall and speed. 119 ·How Transformers Work·★ MEMBER·PAPER·11 min read Build Your Own BPE Tokenizer — Learning Merge Rules, and Getting Punished by Japanese Write the BPE trainer and encoder yourself. Why the artifact of training is an ordered rulebook rather than a vocabulary, how to stop recounting the corpus on every merge, why the first few thousand merge slots in Japanese are spent assembling characters, and how to run a vocabulary-size sweep that actually means something. 120 ·How Transformers Work·★ MEMBER·PAPER·12 min read Build Your Own Mini GPT — A Language Model in 300 Lines Write a character-level GPT in PyTorch from an empty file: tokenizer, causally masked self-attention, training loop, and temperature sampling — then watch Shakespeare's formatting emerge from nothing but next-character prediction. 121 ·Generative Models·★ MEMBER·PAPER·11 min read Build Your Own Diffusion Model — Starting from MNIST A diffusion model built up from nothing on 28×28 handwritten digits: the two conditions a noise schedule has to satisfy, how the step number gets injected into a U-Net, and why the sampler adds noise back at the very end — the places you only discover by writing the code yourself. 122 ·Numerical Computing·★ MEMBER·12 min read Build Your Own Autograd — A Mini PyTorch in 100 Lines Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize. 123 ·Agents·★ MEMBER·PAPER·8 min read Build Your Own Agent Loop — The Minimal Shape of Tool Calling At the center of every AI agent is a single while loop. We build it from scratch without a framework — the shape of JSON function calls, what ReAct actually left behind, and the stopping conditions where nearly every incident originates. 124 ·How Transformers Work·★ MEMBER·PAPER·9 min read Encoder or Decoder — The Fork in the Road Between BERT and GPT One masked triangle in the attention table is what separated BERT from GPT. This piece works through bidirectional versus autoregressive with the equations and an interactive figure, then asks why generation won and where encoders are still the first choice. 125 ·Audio & Speech·★ MEMBER·PAPER·13 min read Representing Sound — Mel Spectrograms and Audio Tokens Why speech models never eat raw waveforms, and what they eat instead: the chain from short-time Fourier transform to the mel scale to the log to discrete tokens. Covers the window-length tradeoff, why MFCCs dropped the DCT, how acoustic and semantic tokens differ, and the config mismatches that silently wreck audio in production. 126 ·Generative Models·★ MEMBER·PAPER·9 min read Music and Audio Generation from Scratch — Sound as Tokens A three-minute song is more than 15 million numbers. The history of music generation is largely the history of folding that sequence down to something a model can hold. Codec tokens, the fork between autoregressive and diffusion approaches, how text conditioning actually bites, and how to reconstruct the shape of Suno-style products from published parts — no prior knowledge assumed. 127 ·Agents·★ MEMBER·PAPER·10 min read Paper walkthrough: ASI-Bench — peeling away human guidance to measure what AI can do alone ASI-Bench keeps the research goal, data and grading fixed while stripping away human methodological guidance one layer at a time. Average scores fall 50.91 → 29.10 → 26.62, and the place where things break is not method selection. 128 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper walkthrough: Apodex 1.1 — scaling agents around completed work Not a bigger model and not more thinking time — Apodex 1.1 scales two other surfaces: the environments an agent learns in, and the way work is organised across agents. A walkthrough from the task contract to the AgentOS delivery gate, the numbers, and the limits. 129 ·Agents·★ MEMBER·PAPER·9 min read AlphaGo from Scratch — The Marriage of Search and Learning Starting from why Go was considered unsolvable for so long, this piece unpacks how the policy network, the value network, Monte Carlo tree search and self-play each cover the others' weaknesses — with the formulas and the code. It closes with what this design handed down to inference-time compute in LLMs. 130 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps. 131 ·Agents·★ MEMBER·PAPER·11 min read Designing Agent Memory — Short-Term, Long-Term, Episodic An LLM remembers nothing. Conversations only look continuous because you re-send the whole history every turn. Building up from zero: the three-layer split, the function that decides what gets recalled, and the part almost nobody designs — forgetting. 132 ·Agents·★ MEMBER·PAPER·11 min read Evaluating Agents — How Benchmarks and Harnesses Are Built An agent score is never a property of the model alone — it is a property of model plus harness plus environment plus grading rule. This piece opens up a single SWE-bench instance, shows why a tiny per-step gap becomes an order-of-magnitude gap over a long horizon, walks the four routes by which the answer leaks into the working environment, and sets out the conditions under which partial credit is safe. 133 ·Probability & Statistics·★ MEMBER·10 min read Hypothesis Testing and A/B Tests — How to Use a p-value, and How People Misuse It Ship the identical page to two groups and the numbers still won't match. A hypothesis test converts that gap into 'how many units of noise is this?' — that number is the p-value. From the null hypothesis up through the three failures that actually ruin experiments: power, peeking, and multiple comparisons. 134 ·Audio & Speech·★ MEMBER·PAPER·12 min read Speech Synthesis from Scratch — From Text to a Voice Speech synthesis invents a waveform tens of thousands of times longer than the handful of characters it starts from. This walks through why naive regression fails (one-to-many and phase), why text → mel spectrogram → waveform became the standard split, and how a few seconds of reference audio is now enough to carry a voice. 135 ·How Transformers Work·FREE·PAPER·10 min read The Transformer, End to End — One Token's Journey from Embedding to Output Between typing a prompt and getting a word back, a single token gets handed a vector, rewritten in room after room, and finally turned back into language. Tokenizing, embedding, position, attention, feed-forward, residuals, and the output head — walked as one continuous trip rather than a pile of parts. 136 ·Databases·★ MEMBER·9 min read Transactions and ACID — Concurrency Hell and the Isolation Levels Why a balance never evaporates when the power dies mid-transfer, and why the bug that only shows up in production is almost always a concurrency bug. Atomicity, isolation levels, MVCC and deadlocks — from zero assumed knowledge down to the actual PostgreSQL and MySQL parameter names. 137 ·Security·★ MEMBER·10 min read TLS from Scratch — Key Exchange and Certificates Behind the padlock icon, your browser pulls off something strange: it agrees on a secret with a stranger, over a wire everyone can read. This walks through key exchange, certificates, and the TLS 1.3 handshake from zero, ending with why a man in the middle cannot win. 138 ·Linear Algebra·FREE·10 min read Tensors and Shape Manipulation — If You Can Read einsum, You Can Read Papers The Σ_j A_ij B_jk in the paper and the x.transpose(1,2) in the code say the same thing, and einsum is the bridge between them. Three tools — axes, broadcasting, contraction — are enough to write attention in a single line. 139 ·Networking·FREE·11 min read From TCP to QUIC — Reinventing Reliable Communication The internet is built on a foundation that makes no promise of delivery. This piece starts from zero: how TCP manufactures reliability out of sequencing, retransmission and congestion control, why HTTP/2 hit a wall, and why QUIC deliberately rebuilt all of it on top of UDP. 140 ·Cloud & Ops·FREE·10 min read Serverless and Cost Design — Cloud That Won't Bankrupt You "Pay for what you use" also means "pay for what someone else uses of you." Starting from zero, this piece builds up scale-to-zero, the GB-second billing unit, and the accident patterns that make a bill grow exponentially — ending with a kill switch you can actually design into your own project. 141 ·Compilers & Runtimes·★ MEMBER·11 min read Why Python Is Slow — Said Precisely Saying Python is slow bundles three unrelated complaints: the cost of a single operation, the way data is laid out in memory, and the fact that CPU work does not spread across threads. Objects and the eval loop, the real reason NumPy is fast, what the GIL actually protects, and how far free-threaded builds since PEP 703 get you — from zero assumed knowledge. 142 ·Data Structures·★ MEMBER·PAPER·13 min read Probabilistic Data Structures — Counting Without Counting Bloom filters, HyperLogLog and the Count-Min sketch explained from zero — how giving up the right to always be correct buys you memory that never grows, and how large services actually operate these sketches. 143 ·Large Language Models·★ MEMBER·PAPER·10 min read Building a Pretraining Corpus — From Web Sludge to Textbook Quality Behind the single line "pretrained on a large corpus of web text" sit four stages: text extraction, quality filtering, deduplication, and mixture weights. This is a from-scratch walkthrough of how a gravel heap called Common Crawl gets sifted into textbook-quality prose — from the MinHash equation to the parameter names you actually touch. 144 ·OS & Runtime·FREE·13 min read Processes and Memory from Scratch — What Is the OS Actually Protecting? One app can crash without taking the machine down with it. Process isolation and virtual memory are what make that ordinary. Starting from a post-office-box analogy, this builds up paging, address translation, TLBs, page faults, swap and the OOM killer from zero — and ends with you able to read free and dmesg yourself. 145 ·Cloud & Ops·★ MEMBER·10 min read Observability — Logs, Metrics, and Traces in Practice When the phone rings at 3 a.m., can you answer without logging into production and looking around? Starting from zero, this piece builds up structured logging, metric cardinality, trace context propagation, and SLOs with error budgets — then lines them all up as a single motion: notice, narrow, confirm. 146 ·Security·★ MEMBER·PAPER·9 min read LLM Security — Prompt Injection and How to Actually Defend Against It A language model cannot tell your instructions apart from words printed on a document it was asked to read. This article works from that single fact to direct injection, indirect injection, the tool boundary that decides your blast radius, and the layered defence that actually holds. 147 ·Compilers & Runtimes·★ MEMBER·10 min read JIT and GC — Getting Faster While Running, Cleaning Up While Running A just-in-time compiler and a garbage collector are both services that have to work without stopping the application. Hot spot detection, tiered compilation, inlining, speculation and deoptimization, generational GC, tri-color marking and write barriers — built up from zero, ending where you can read a GC log and a JIT log yourself. 148 ·Networking·★ MEMBER·8 min read HTTP/1.1 → 2 → 3 — The Road to Multiplexing Thirty years of HTTP is really one argument: how many conversations can share a single connection? Why 1.1's one-at-a-time rule created traffic jams, what HTTP/2 solved with frames and streams, why the jam simply moved one floor down, what HTTP/3 separated by swapping the foundation — and how to measure your own network before picking one. 149 ·RAG & Retrieval·★ MEMBER·PAPER·10 min read GraphRAG from Scratch — Where Knowledge Graphs Meet Retrieval Rebuild your documents as a web of entities and relationships and you can answer questions ordinary RAG cannot reach — the ones about the corpus as a whole. Extraction, entity resolution, community summarization, and local/global search from first principles, ending with an honest account of when it's overkill. 150 ·Cloud & Ops·★ MEMBER·10 min read The Economics of GPU Cloud — Rent, Buy, or Commit The same GPU carries four prices at once. Put on-demand, committed, spot, and owned hardware on a single equation: the break-even utilization, the true cost of spot once interruptions are priced in, the commitment burn rate, and the line items that never appear on the rate card. 151 ·Model Families·★ MEMBER·9 min read The GPT Lineage — Design Thinking from GPT-1 to Today A generation-by-generation walk from GPT-1 to GPT-4o and the reasoning models, asking what changed and what deliberately did not. At the centre sits one machine that only ever predicts the next token; what moved was scale, the order of training, and how the output was disciplined. 152 ·VLMs & Multimodal·★ MEMBER·PAPER·10 min read Document AI and OCR Today — How an LLM Ends Up Reading Your Invoices How machines came to read invoices and scanned PDFs, from first principles: text detection and CTC, how errors are measured (CER), LayoutLM's trick of embedding coordinates alongside words, the OCR-free Donut line, and today's habit of handing the page straight to a VLM — plus the walls that matter in production: tables, handwriting, and hallucination. 153 ·Networking·★ MEMBER·11 min read DNS and CDNs — What Happens Between Enter and Pixels In the second or so between pressing Enter and seeing a page, a name gets resolved down a tree, four layers of cache get consulted, and an edge server somewhere near you answers instead of the origin. Here is how DNS walks that tree, what a TTL does not promise, and how a CDN talks its way around the speed of light — from zero background. 154 ·Probability & Statistics·FREE·13 min read A Field Guide to Probability Distributions — Where Normal, Poisson, and the Exponential Family Come From The normal and Poisson distributions aren't formulas to memorize — they're the shapes that certain situations inevitably produce. Follow three stories (adding things up, counting rare events, and refusing to assume more than you know) and the textbook zoo collapses into one river called the exponential family, with sigmoid and softmax falling out of it. 155 ·Databases·FREE·10 min read Database Internals — What Happens Behind a Single Line of SQL Between hitting Enter on a SELECT and getting rows back, five stages run inside the database: parser, rewriter, planner, executor, storage. Here is why two plans returning identical rows can differ by orders of magnitude, and why stale statistics make a fast query go slow overnight — from zero background to the knobs you actually turn. 156 ·Training & Alignment·★ MEMBER·PAPER·11 min read Building a Dataset in Practice — Collect, Clean, Blend Most of the work in building a model is really the work of building its dataset. From designing the population you sample from, to calibrating filter thresholds, measuring leakage into your test set, converting mixture ratios into epochs, using synthetic data where it belongs, and writing annotation guidelines that actually hold — explained from zero, with the formulas and the procedures. 157 ·Security·FREE·13 min read Cryptography from Scratch — Symmetric Keys, Public Keys, and Hashes Cryptography isn't about producing unreadable text — it's about engineering an extreme gap in effort between people who hold the key and people who don't. We build up the three tools (symmetric, public-key, hashing) from zero, explain why factoring shows up at all, what a signature actually promises, and how every widely deployed cipher has eventually broken. 158 ·OS & Runtime·★ MEMBER·12 min read Containers from Scratch — What namespaces and cgroups Actually Are A container is not a lightweight virtual machine — it is an ordinary process with its view restricted. We take it apart into namespaces (what you can see), cgroups (how much you may use), and images (stacked read-only layers), from the first analogy through overlayfs, cpu.max, and how to read exit code 137. 159 ·Compilers & Runtimes·FREE·11 min read Compilers From Scratch — How Source Becomes Machine Code One line of source code, followed all the way down through lexing, parsing, semantic analysis, intermediate representation, optimization, and code generation, until it comes out as a single machine instruction. Along the way: why bugs surface at -O2, and why your benchmark loop disappears. 160 ·Security·★ MEMBER·14 min read Authentication and Authorization — From Passwords to OAuth and Passkeys "Who are you?" and "what are you allowed to do?" are two different questions. This piece builds up password storage, sessions and tokens, the four actors in OAuth, and why passkeys resist phishing — assuming nothing to start with. 161 ·Complexity·★ MEMBER·12 min read Approximation Algorithms — Trading Exactness for a Guarantee The craft of giving up on the optimal answer while attaching a price tag that reads "never worse than X times optimal". We build up the approximation ratio, carry a greedy proof all the way to the end, and see why the triangle inequality flips the entire conclusion for the traveling salesman problem. 162 ·Fabrication & Packaging·★ MEMBER·11 min read Yield and Design — DFM, the Art of Giving Something Up A circuit can be correct in simulation and still fail to make money in volume. Defect density models and the clustering factor, design rules that restrict what shapes you may draw, redundancy that assumes breakage, and binning that turns one die into several products — the engineering of trading performance for yield, from first principles. 163 ·Deep Learning Basics·★ MEMBER·PAPER·15 min read Weight Initialization and Regularization — What Lets Training Start, and What Keeps It Going The same architecture will train or refuse to train depending on nothing more than how large the random numbers in its weights were. Starting from how variance propagates, this piece derives Xavier and He initialization, then turns to weight decay and dropout as the conditions that keep training going — with interactive figures, PyTorch code and the mistakes that actually cost people days. 164 ·Time Series·★ MEMBER·PAPER·11 min read Time-Series Anomaly Detection — The Math Behind the Alerts An alert should fire on the gap between what you observed and what that moment predicted — not on the raw size of a number. Four rulers for measuring that gap (robust statistics, forecast residuals, subsequence distance, changepoints), built up from analogy to formula, then the two things that actually break in production: how you pick the threshold and how you evaluate. 165 ·Scaling & Power·★ MEMBER·11 min read Thermal Design from Scratch — The Wall in 3D Stacking Is Heat Nearly all the power that enters a chip leaves it as heat. Temperature is set by a series stack of thermal resistances, and once you stack dies vertically the hottest layer heats up with the square of the layer count. Heat flux and hotspots, the lag that thermal mass introduces, and which term liquid cooling actually removes — picking up where the power budget left off. 166 ·Computer Architecture·★ MEMBER·PAPER·10 min read Systolic Arrays — Building the Heart of the TPU From Scratch Piling on multipliers doesn't make a chip faster, because moving data costs more than the arithmetic does. This piece takes the grid at the center of the TPU — the systolic array — and shows exactly how it maps the triple loop of matrix multiplication onto rows, columns and clock ticks, from a 2×2 hand trace to a cycle-accurate simulator to the batch-size and shape rules you'll actually tune. 167 ·Media in Production·FREE·10 min read HLS and DASH — How Video Actually Reaches You Streamed video does not flow — a player downloads small files a few seconds long, one after another. What segments and manifests really are, the arithmetic behind how a player picks a rendition, where latency comes from, how low-latency modes cheat it, and why none of it works without a CDN. 168 ·Audio Codecs·★ MEMBER·PAPER·11 min read Why Opus Won — The Design of a Modern Audio Codec Audio codecs used to split cleanly into speech codecs and music codecs. Opus ended the split by stacking two engines, SILK and CELT, inside a single bitstream. This piece works up from zero: how linear prediction and the MDCT divide the spectrum between them, why CELT refuses to let band energy collapse, what actually sets a codec's latency, and the technical and institutional reasons Opus became WebRTC's mandatory codec. 169 ·Complexity·FREE·9 min read NP-Completeness from Scratch — Not Unsolvable, but Fast to Verify NP does not stand for Non-Polynomial. It is the class of problems where, if someone hands you an answer, you can check it quickly. We build up P vs NP, reductions and NP-completeness from zero, then look at how all of it shows up in shift rosters and delivery routes. 170 ·Video Codecs·★ MEMBER·PAPER·11 min read Neural Compression — The Codec That Learns JPEG's quantization table and H.264's prediction modes were tables people worked out by hand. Neural compression hands that design job to the data instead. The two pillars — an autoencoder and an entropy model — the trick that gets around non-differentiable rounding, the hyperprior, and what standards bodies are actually arguing about on the road to JPEG AI. 171 ·Audio Codecs·★ MEMBER·PAPER·12 min read Neural Audio Codecs — EnCodec and the Foundation Under Speech LLMs MP3 and Opus were hand-designed around what the ear cannot hear. Neural audio codecs learn the coding itself and turn sound into a finite alphabet of integers. From vector quantization to residual VQ, EnCodec's bitrate arithmetic, and why VALL-E and AudioLM are built on top of it — starting from zero. 172 ·Information Theory·★ MEMBER·PAPER·11 min read Mutual Information — Putting a Number on What You Know How much does knowing one thing shrink your uncertainty about another? Mutual information turns that into a single number. We build it from entropy subtraction, work through conditional MI, the data processing inequality and why estimation is hard, and end at why the InfoNCE loss behind modern contrastive learning is called a lower bound on mutual information. 173 ·Image Codecs·★ MEMBER·10 min read WebP, AVIF, JPEG XL — The Image Codec Changing of the Guard Three successors clearly beat JPEG on efficiency, and the web is still full of JPEGs. This piece traces where each newcomer came from, why still-image codecs keep getting born out of video codecs, and the structural reason a lingua franca doesn't die just because something faster shows up — down to how you actually compare codecs and ship the files. 174 ·Search & Optimization·★ MEMBER·11 min read Simulated Annealing and Genetic Algorithms — What to Do When Exact Solving Breaks Down Why search methods that guarantee nothing end up running real delivery routes and factory schedules. From the three ingredients of local search, through temperature in annealing and populations in genetic algorithms, to the harder question of when you should not reach for them at all. 175 ·Calculus & Optimization·★ MEMBER·11 min read Matrix Calculus from Scratch — Derive the Backward Pass Yourself Where does the transpose in ∂L/∂W = XᵀG actually come from? Matrix calculus is not a formula sheet to memorize — it is one move: rotate dX to the right inside a trace. From denominator layout and shape-checking to the gradients of a linear layer and softmax + cross-entropy, ending with a double-precision gradient check. 176 ·Search & Optimization·★ MEMBER·11 min read Linear Programming from Scratch — The Workhorse of Optimization The oldest and most widely deployed tool for choosing the best option under limited stock, budget and time. From the three-part recipe for writing a model down, to why the answer always sits at a corner, to what duality tells you a kilo of flour is really worth — built up from zero. 177 ·Accelerators·★ MEMBER·9 min read The Inference Chip Wars — Inside the Design Philosophies of Groq, Cerebras, and the LPU Inference-only chips exist because decoding is limited by reading, not by arithmetic. From zero background: the two answers — put everything in SRAM (Groq/LPU) and make the chip a whole wafer (Cerebras) — the software counterattack, and the arithmetic you need to read the market without swallowing vendor benchmarks whole. 178 ·Device Physics·★ MEMBER·9 min read From FinFET to GAA — Why the Transistor Had to Go Vertical Why did a flat transistor grow a fin, and then become a stack of thin slabs? With three tools — short-channel effects, DIBL, and the natural length — this piece shows that FinFET and GAA nanosheets were not a matter of taste but a corner physics painted the industry into. No prior knowledge assumed. 179 ·Model Families·FREE·9 min read The DeepSeek Family from Scratch — Breaking In with MoE and Distillation DeepSeek redrew the open-LLM map with four tools: MoE, MLA, GRPO, and distillation. Walk the V2/V3/R1 lineage from zero prerequisites, learn how to read that famous training-cost number, and find out what the distilled models are actually for. 180 ·Machine Learning Basics·FREE·PAPER·11 min read Decision Trees and Gradient Boosting — Still the Champion on Tabular Data On data made of rows and columns, ensembles of decision trees are still the first thing to reach for. From the intuition behind a single split to boosting, what XGBoost and LightGBM actually do, and why neural networks have not taken this territory — built up from nothing. 181 ·Computer Architecture·★ MEMBER·PAPER·10 min read CPU Pipelines and Branch Prediction — The Factory Inside One Clock Tick An instruction appears to finish in a single clock tick because the machine is an assembly line. From the five classic stages, through the three kinds of hazard, to branch prediction, speculative execution, and the day the leftovers of speculation turned into an information leak — Spectre. 182 ·RAG & Retrieval·★ MEMBER·10 min read Chunking Strategies — How You Split Decides What You Can Find Most of a RAG system's quality is decided by how you split documents. The arithmetic of fixed-size chunks and overlap, structural and semantic splitting, parent-child chunks, and the two things that break every splitter: tables and equations. 183 ·Scaling & Power·★ MEMBER·10 min read The Economics of Chiplets — We Split Dies Because We Cannot Build Them Big Chips are split into chiplets not because splitting is faster but because a single large die cannot be built at a price anyone will pay. The reticle limit, the exponential in yield, cost per good die, the point where splitting starts to lose money, mixing process nodes, and UCIe as a standard for the seam. 184 ·Data Structures·★ MEMBER·10 min read B-Trees and LSM-Trees — The Heart of Every Database Almost every database ever shipped sits on a B-tree or an LSM-tree. Starting from one physical fact — storage can only be written a page at a time — here is why the two designs came out opposite, what write amplification really is, and how PostgreSQL and RocksDB differ, from zero background to the knobs you actually turn. 185 ·Device Physics·FREE·11 min read Band Theory from the Ground Up — Why It Had to Be Silicon What actually separates a conductor from an insulator? Starting from why atomic energy levels smear into bands, this piece builds up bandgaps, Fermi statistics and doping with no prior physics assumed — and lands on the real answer to 'why is every chip made of silicon', which turns out to be about the oxide, not the element. 186 ·Numerical Computing·FREE·10 min read How Autodiff Actually Works — Unpacking the PyTorch Magic Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine. 187 ·Audio & Speech·FREE·11 min read Speech Recognition from Scratch — From Waveform to Text How a stream of numbers from a microphone becomes words, starting from zero: spectrogram features, the alignment problem that CTC solved, autoregressive encoder-decoder models, and Whisper — in the order history solved them. 188 ·Deep Learning Basics·FREE·PAPER·13 min read Activation Functions from Scratch — Why Nonlinearity Is Non-Negotiable Without an activation function, a hundred stacked layers can do exactly what one layer does. Starting from that one-line proof, this article traces why sigmoid was abandoned, why ReLU won, and why today's LLMs settled on SiLU and SwiGLU — with an interactive plot where you can drag the input and watch the slope. 189 ·Paper Deep-Dives·★ MEMBER·PAPER·13 min read Paper Deep Dive — Large Discovery Models: giving an LLM a value signal for what to try next An LLM can propose candidates but cannot reliably grade its own proposals. Large Discovery Models tilts the LLM's proposal distribution by an acquisition value drawn from a Gaussian process surrogate, so search keeps working when every evaluation is expensive. From the equations to the reported numbers. 190 ·Agents·★ MEMBER·PAPER·13 min read Paper walkthrough: Zetta ζ — a robot harness that repairs itself mid-execution, with the policy frozen Existing embodied agents only reflect once an episode is over. Zetta instead runs code-based critics at action frequency and intervenes the moment a failure signature appears — without touching a single policy weight. A ground-up walkthrough of the design, following the paper itself. 191 ·Video Codecs·★ MEMBER·10 min read Motion Compensation from Scratch — Where 90% of Video Compression Happens What decides where a video codec's bits go isn't the transform or the quantizer — it's the prediction at the front door. What block matching actually minimizes, why motion vectors are not object motion, and how P/B reference structures and GOP boundaries turn into real streaming failures. 192 ·Generative Models·FREE·10 min read VAEs from Scratch — Stir Probability into "Compress and Restore" and You Get a Generator An autoencoder that only compresses and restores cannot invent anything new. This walks through why a single drop of probability turns it into a generative model — ELBO, the reparameterization trick, and walking the latent space — assuming no prior knowledge. 193 ·Scaling & Power·FREE·10 min read What Moore's Law Actually Says — What Ended, and What Is Still Going "Double every two years" was never a law of physics — it was a timetable the industry agreed to keep. Dennard scaling is what ended; cost is what slowed; density and going vertical are what continue. A ground-up tour through log axes, the flat-to-FinFET-to-GAA story, and the arithmetic of cost per transistor. 194 ·How Transformers Work·FREE·PAPER·10 min read Tokenizers from Scratch — The Unit an LLM Cuts the World Into An LLM reads neither characters nor words. How BPE builds a vocabulary, what SentencePiece actually fixed, why some languages pay more for the same sentence, and what you trade away when you grow the vocabulary — worked by hand and in code, from zero. 195 ·Time Series·★ MEMBER·PAPER·11 min read Do Transformers Actually Work on Time Series? — The Argument and the Practical Answer In 2022 a single linear layer beat the whole crop of time-series Transformers on the standard benchmarks. The culprit turned out to be tokenization, not attention — a diagnosis PatchTST fixed by patching and iTransformer by transposing the axes. And yet gradient boosting keeps winning the practitioner competitions, for reasons that come down to the shape of real data. 196 ·Agents·★ MEMBER·PAPER·11 min read Paper walkthrough: StateM — 95.3% on Terminal-Bench 2.1 and a USD 15 run, without touching a single weight Long-horizon agents fail even when the model underneath can solve every individual step. StateM leaves the weights alone and hardens only the execution system around the agent, reporting 95.3% on Terminal-Bench 2.1 and a final scoring run billed at roughly USD 15 instead of USD 574.68. A ground-up walkthrough of the harness-scaling bet. 197 ·Agents·★ MEMBER·PAPER·13 min read Paper Explainer: SemaPLC — The Agent That Isn't Allowed to Say "Done" Most work on LLM-generated factory control code stops at showing that the code can run. SemaPLC forbids the agent from declaring completion until logged external checks confirm it, lifting live-runtime behavior from a baseline ceiling of 31.4 to 52.2. 198 ·Generative Models·★ MEMBER·PAPER·8 min read CFG and Samplers — What the "Strength" Knob in Generative AI Really Does What "CFG Scale" and "Sampling steps / method" actually do inside an image generator, explained from zero. CFG extrapolates along the difference between the conditional and unconditional prediction; the sampler is a numerical ODE solver. Both knobs trade something away. 199 ·RAG & Retrieval·★ MEMBER·PAPER·10 min read Recommenders and Embeddings — Same Math as RAG, Different Goal What sits behind "recommended for you" is very nearly the same math as RAG's vector search. A from-zero tour: matrix factorization, two-tower models, and how the ANN stack is reused — plus why the evaluation and the failure modes end up completely different. 200 ·Complexity·★ MEMBER·12 min read Randomized Algorithms — Why Rolling Dice Makes Things Faster Why does adding randomness make an algorithm faster? We build up quicksort with a random pivot, the one-sided error of Bloom filters, and the Monte Carlo / Las Vegas split from zero background — ending with the ways mishandled randomness actually breaks systems in production. 201 ·Audio Codecs·★ MEMBER·10 min read MP3 and Psychoacoustics — The Science of Sounds You Cannot Hear MP3 does not throw away sounds that aren't there — it throws away sounds that are there and inaudible. This piece works through the three holes in human hearing (absolute threshold, simultaneous masking, temporal masking) down to the equations, then follows SMR into bit allocation, the encoder's two nested loops, and the bit reservoir. 202 ·Calculus & Optimization·★ MEMBER·PAPER·12 min read Beyond SGD — Adam, Second-Order Methods, and Constrained Optimization What exactly is momentum accumulating? What does each of Adam's four lines do? What did AdamW fix? And why does nobody train an LLM with second-order methods that are supposedly faster? Metaphor, equations, live figures, code, and production practice — no prerequisites assumed. 203 ·Numerical Computing·★ MEMBER·10 min read Numerical Pitfalls — Cancellation, Rounding, and logsumexp Where "the loss went nan three hours into the run" actually comes from, built up from nothing: how rounding enters, how the condition number amplifies it, and why subtracting two close numbers is so destructive. It all converges on logsumexp — the one trick sitting inside every softmax and cross-entropy implementation. 204 ·Deep Learning Basics·★ MEMBER·PAPER·10 min read A History of Normalization Layers — From BatchNorm to RMSNorm The layer that made deep learning actually deep, explained from zero. The internal-covariate-shift controversy behind BatchNorm, why LayerNorm threw away the batch axis, and why every modern LLM converged on RMSNorm. 205 ·Training & Alignment·★ MEMBER·PAPER·10 min read Mixed Precision Training — Going Faster in fp16/bf16/fp8 Without Breaking Halve the bits and training gets faster — right up until your gradients quietly turn into zeros or infs. We start from the two ways it breaks, work out what loss scaling actually does, why bf16 made it unnecessary, and what came back with fp8. Ends with the PyTorch AMP knobs and the clipping-order mistake that fails silently. 206 ·Linear Algebra·★ MEMBER·13 min read A Tour of Matrix Decompositions — When to Reach for LU, QR, Cholesky, or SVD A decomposition rewrites one awkward transformation as a product of parts you can actually work with: solve it (LU), solve it at half price (Cholesky), orthogonalize and fit (QR), or survive a broken matrix (SVD). Starting from why nobody computes an inverse, this piece builds up to a single table that tells you which one to reach for, plus the numpy that goes with it. 207 ·Probability & Statistics·★ MEMBER·11 min read Markov Chains from Scratch — The Process That Only Looks at Now What happens next depends only on where you are now — that single act of forgetting is a Markov chain. From transition matrices and stationary distributions to why PageRank is an eigenvector and why MCMC gets to ignore the normalizing constant, built from nothing assumed. 208 ·Training & Alignment·★ MEMBER·PAPER·10 min read Learning Rate Schedules — Why Warmup and Why Cosine The learning rate is not a fixed number — it is a curve you design across the whole run. Why we deliberately start slow (warmup), why we come down along a cosine, and what else has to move when batch size changes. Equations, live figures, PyTorch code, and the mistakes that actually break runs. 209 ·Computer Architecture·★ MEMBER·PAPER·9 min read The GPU Memory Hierarchy — HBM, SRAM, Registers, and Why Movement Wins What sets a GPU's speed is not the arithmetic units but where the data sits — registers, shared memory, L2 or HBM — and how many times it is moved. Capacities and bandwidths by order of magnitude, arithmetic intensity and tiling, a roofline per level of the hierarchy, and finally FlashAttention: more FLOPs, less time. 210 ·Generative Models·★ MEMBER·PAPER·10 min read The Rise and Fall of GANs — An Invention Trained by Rivalry, and Why Diffusion Won In 2014, GANs upended image generation with one idea: don't write down what 'realistic' means — learn it. This traces the decade that followed, through mode collapse, the Wasserstein rescue, and the reasons diffusion models eventually took the crown. 211 ·Generative Models·★ MEMBER·PAPER·10 min read Flow Matching from Scratch — What Came After Diffusion, and Why It Goes Straight Flow matching — the formulation underneath Stable Diffusion 3 and FLUX — explained with no prerequisites. Velocity fields, ODEs, and rectified flow, told through analogy and interactive figures, all the way to the scheduler settings you actually touch in production. 212 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough — FACET: Grounding Instruction, Environment, Solution and Verifier in One Executable State A terminal task is a four-part bundle: instruction, environment, reference solution, verifier. FACET builds and runs the environment first, then uses that realized state as shared ground for every other artifact. From 71K agent skills it produces 6,078 validated tasks, and 1.2K successful trajectories lift Qwen3.5 at 4B, 9B and 27B alike. 213 ·Fabrication & Packaging·★ MEMBER·10 min read EUV Lithography — The Madness of Making 13.5nm Light Light at 13.5nm is absorbed by air and by every lens material there is. So why did the semiconductor industry build an entire industry around it? Tin plasma sources, optics with no lenses at all, and the cost arithmetic against multi-patterning — explained from zero. 214 ·Coding Theory·FREE·11 min read Error Correction from Scratch — Sending Data on the Assumption It Will Break Starting from a single parity bit, this article builds up Hamming distance, syndrome decoding and Reed–Solomon codes with no assumed background, then explains why a smudged QR code still scans — and where ECC bites you in production. 215 ·Agents·★ MEMBER·PAPER·9 min read Paper Explained: EnvHarness — Reshaping an Agent's Training World Without Rebuilding It The environments LLM agents train in are hand-built and frozen: blind to the agent's weaknesses and left behind as it improves. EnvHarness wraps a static environment in plug-in components that reshape its behavior without touching the underlying logic — and keep the original verifier intact. 216 ·Information Theory·FREE·9 min read Entropy and Cross-Entropy — Where the Loss Function Comes From The cross-entropy loss you write in every classifier wasn't handed down by decree. Starting from the idea of surprise, we walk a single path through entropy and cross-entropy to that familiar one-liner — and finish with why squared error is the wrong tool for the job. 217 ·Agents·★ MEMBER·PAPER·9 min read Paper Explainer: Why Agent Skills Work — and Where They Break Handing an agent a written procedure improves its scores. This paper dissects why, using 8,135 trial records and 528 matched trajectory triples. Skills work as procedural anchors rather than knowledge injection — and they break once the skill library grows. 218 ·Device Physics·★ MEMBER·10 min read Reading Chip Design as a Power Budget — The Physics of Leakage and Heat A modern chip is limited not by what fits, but by how much of it can be lit at once. This piece reads that limit as a household budget: dynamic power versus leakage, the positive feedback that makes leakage swell with temperature, the thermal resistance that converts watts into degrees, and the three — and only three — ways a designer can spend the watts. 219 ·Data Structures·★ MEMBER·11 min read Cache-Friendly Code — Why Two O(n) Loops Can Differ by 10× Two implementations with identical complexity can differ by an order of magnitude, because the CPU never fetches one value — it fetches a 64-byte block. Locality, cache lines, arrays versus linked lists, AoS versus SoA, loop order and false sharing, from zero assumed background to checking it yourself with perf. 220 ·Probability & Statistics·★ MEMBER·11 min read Thinking Bayesian — A Working Feel for Priors, Likelihoods, and Posteriors Bayesian updating is the act of feeding yesterday's posterior back in as today's prior. With a conjugate prior the whole update collapses into adding pseudo-counts, and an A/B test becomes two numbers: the probability of winning and the expected loss. From zero background to the traps in stopping rules and prior choice. 221 ·How Transformers Work·★ MEMBER·PAPER·9 min read A Field Guide to Attention Variants — MQA, GQA, Sliding Windows, Linear Attention MQA, GQA, sliding windows and linear attention are not four unrelated tricks. One multiplication decides how large a KV cache gets, and every variant is a decision about which factor in it to attack. Lined up against that formula, the family tree shows exactly what each one gave up and what it bought. 222 ·Inference & Serving·★ MEMBER·PAPER·15 min read Paper Explained: Agentic ESOpt — Drop Backprop, Jiggle the Weights, and Train Long-Horizon LLM Agents No gradients at all: spawn G perturbed copies of the model, run them in the environment, and nudge the weights toward the ones that scored well. That buys full-parameter updates of a 27B agent at inference-level memory (8.41GB) — and a 12.50-point lead over GRPO on 15-move Sudoku. A ground-up walkthrough of the NUS-led paper. 223 ·Fabrication & Packaging·★ MEMBER·10 min read Advanced Packaging — How CoWoS and HBM Stacking Became the Bottleneck for AI What limited the supply of AI accelerators was not leading-edge logic wafers but the step that puts several finished chips into one part. From the reticle limit and the exponential math of yield, through 2.5D interposers, HBM stacks and TSVs, to hybrid bonding and the bill that heat and warpage send back. 224 ·Linear Algebra·FREE·7 min read The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read. 225 ·Large Language Models·★ MEMBER·PAPER·11 min read Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper. 226 ·VLMs & Multimodal·★ MEMBER·PAPER·8 min read How VLMs Came Together — Wiring a Vision Encoder into an LLM A vision-language model is three parts: a vision encoder, a projection layer, and an LLM. From the bridge CLIP built, through the three schools of connector design, to resolution strategies and the implementation constraints that bite in practice. 227 ·Linear Algebra·★ MEMBER·PAPER·10 min read Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters. 228 ·Large Language Models·★ MEMBER·PAPER·9 min read Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't) Scaling laws turn 'bigger is smarter' into an equation. This article walks from the Kaplan-era recipe to the Chinchilla paper (Hoffmann et al., 2022) — 400+ training runs, three independent estimation methods, and the 70B-vs-280B showdown — using only the paper itself as evidence. 229 ·Agents·★ MEMBER·PAPER·13 min read Paper Deep-Dive: Recursive Synthesis — Extending Verified Tasks Into 40,000 Long-Horizon Terminal Problems Long-horizon terminal tasks cost hundreds to thousands of dollars each to author. RST mass-produces them for about $0.05 by recursively extending already-verified seeds. Fifteen rounds yielded 37,484 tasks, with oracle solutions growing from a median of 67 lines to 374, and the resulting trajectories genuinely improved Qwen3.5 under SFT and PPO — a walkthrough of the data-synthesis paper from Tencent and collaborators. 230 ·Model Families·★ MEMBER·7 min read The Qwen Family from Scratch — Why It Tops Hugging Face's Download Charts Alibaba Cloud's Qwen is a fixture at the top of Hugging Face's download charts. From Qwen2.5's every-size-plus-specialist catalog strategy to Qwen3's thinking mode and MoE, this guide covers the lineage, why it excels at multilingual and code work, and the pitfalls that bite in production — assuming zero prior knowledge. 231 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: No Gold Answers, No Stronger Teacher — How u-OPSD Distills From Its Own Majority Vote u-OPSD (arXiv:2608.06296) replaces the gold solution in on-policy self-distillation with the model's own majority vote, then corrects only the rollouts that disagree with it. A from-scratch walkthrough grounded solely in the paper. 232 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: Metis — A 'Memory Foundation Model' That Moves Agent Memory Inside the Model Agent memory today is mostly bolted on from the outside via RAG. This paper proposes memory foundation models — models whose forward pass natively stores, forgets, and updates information — and builds Metis, the first prototype. A from-scratch walkthrough of how it works, how well it works, and where it breaks. 233 ·Agents·★ MEMBER·PAPER·10 min read Paper Walkthrough: MerchantBench — Can an LLM Agent Run an Online Store for a Year? Why It Earns Only 27.3% of What Humans Do A close read of MerchantBench, which measures the long-term coherence of LLM agents across a 365-day simulation of running an online store. Why the best configuration reaches only 27.3% of the average human's final net worth, read through three failure patterns: decaying activity, premature withdrawal, and policies frozen against the evidence. 234 ·Agents·★ MEMBER·PAPER·8 min read Paper Explained: LongHorizon-Harness — Long-Horizon Agent Tasks Are a State-Management Problem, Not an Execution Problem Without touching the model at all, keeping task state outside execution and running a Manage-Execute-Audit loop dramatically improves long-horizon task performance — a walkthrough of Alibaba DreamX team's harness-design paper, from analogy to pseudocode. 235 ·Information Theory·★ MEMBER·PAPER·9 min read KL Divergence From Scratch — Measuring the Gap Between Two Distributions KL divergence measures the gap between two probability distributions. We build it up from a compression metaphor to the definition, its famous asymmetry, and a numpy implementation — then watch it at work as the regularizer in VAEs and the leash in RLHF. 236 ·Data Structures·★ MEMBER·PAPER·9 min read Hashing and Nearest-Neighbor Search — The Groundwork Under Vector Search Two inventions that made looking things up fast — the exact-match hash table, and LSH and HNSW for searching by meaning — from zero assumed knowledge. What is actually running underneath RAG and every vector database. 237 ·Search & Optimization·★ MEMBER·PAPER·8 min read Graph Algorithms from Scratch — Shortest Paths and Where They Lead From transit apps to vector search, the world runs on dots and lines. We build up BFS, Dijkstra, and A* assuming zero background, then follow one unbroken thread all the way to HNSW — the graph search powering retrieval in the LLM era. 238 ·Agents·★ MEMBER·PAPER·11 min read Paper Deep-Dive: Frontis-MA1 — Training the AI That Builds AI: One Step Toward Recursive Self-Improvement in ML Engineering Instead of training a model to solve tasks, train the operations that improve solutions — then run evolutionary search with those trained operations. A walkthrough of the OpenMLE stack and the 35B Frontis-MA1 model, straight from the paper: what's behind the 39.39%→71.21% jump on MLE-Bench Lite, how the search got cheaper and better at once, and the limits the authors themselves spell out. 239 ·Numerical Computing·★ MEMBER·PAPER·9 min read The FFT from Scratch — Why Convolution Turns into Multiplication A from-zero walk through the Fourier transform: a smoothie metaphor, the spinning-needle intuition, the DFT formula, and the divide-and-conquer trick behind the FFT — ending with the polynomial-multiplication view that makes the convolution theorem feel obvious. 240 ·RAG & Retrieval·FREE·PAPER·6 min read Embeddings from Scratch — from word2vec Intuition to Contextual Embeddings Why does 'king − man + woman ≈ queen' actually work? A from-zero tour of embeddings: the map metaphor, the distributional hypothesis, the word2vec math, numpy code, contextual embeddings after BERT, and the knobs you touch in real RAG systems. 241 ·Calculus & Optimization·★ MEMBER·PAPER·9 min read Convexity and Optimization — Why Deep Learning Works Even Though It Isn't Convex Optimization textbooks teach a stark divide: convex problems are solvable, non-convex ones come with no guarantees. So why does deep learning — whose loss surface is provably non-convex — work at all? From convex sets and functions to saddle points and flat minima, this article connects the whole story in the language of landscapes. 242 ·Complexity·★ MEMBER·8 min read When Big-O and Your Benchmarks Disagree — Caches, Branches, and Memory Bandwidth Two O(n) programs can differ by orders of magnitude in the real world. This article unpacks what Big-O deliberately throws away — cache hierarchies, branch prediction, and memory bandwidth — and how to reason about each. 243 ·Agents·★ MEMBER·PAPER·11 min read Paper Walkthrough: ComBodied Agents — Moving an Agent's Target from Software and Matter to the Person Digital agents rewrite software state; embodied agents rewrite physical state. So who models the person? Combodied Agents bind perception, longitudinal memory, a Personal World Model, and an intervention policy into one closed loop, and move the success criterion from task completion to whether the user's agency survived. A position paper with no experiments — here is its skeleton, and its limits. 244 ·Agents·★ MEMBER·PAPER·9 min read Paper Explained: Co-Evolution in Agentic Systems — Three Stages Toward Self-Directed Evolution Why do agents that are supposed to keep improving after deployment hit a ceiling? A ground-up walkthrough of a survey that organises the field into three stages — evolving peers, evolving environments, and an evolving evolution mechanism — with the defining equations, representative methods, and the open problems in evaluation and safety. 245 ·Fabrication & Packaging·★ MEMBER·9 min read How Chips Are Made — From Wafer to Yield Every GPU and phone processor starts as sand and passes through hundreds of printing steps. From the basic moves — lithography, etching, ion implantation — to the brute force of EUV and the yield math that makes big dies exponentially expensive, explained from zero. 246 ·Probability & Statistics·★ MEMBER·PAPER·10 min read Bayes' Theorem in AI — Priors, Posteriors, and Uncertainty One line of math — Bayes' theorem — turns into three workhorse tools in real AI systems: probability calibration, active learning, and Bayesian optimization. Starting from a positive medical test, we build up priors, posteriors, and uncertainty with zero background assumed. 247 ·Accelerators·★ MEMBER·PAPER·8 min read How AI Accelerators Are Designed — What Actually Separates GPUs, NPUs, and TPUs GPUs, TPUs, and NPUs are three different answers to one question: how much generality do you trade away for matrix multiplication? A from-scratch tour of systolic arrays, dataflow design, and the co-evolution of hardware and quantization. 248 ·Training & Alignment·★ MEMBER·PAPER·11 min read Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself. 249 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper Walkthrough: WorldClaw — Agents That Build Walkable, Editable 3D Open Worlds from a Single Sentence Type 'a tropical pirate stronghold' and get a large-scale 3D world you can walk through — with every building still an editable, standalone asset. A walkthrough of Tencent Hunyuan's WorldClaw, from its plan → terrain → regional-objects pipeline to the render-and-inspect loops where agents check their own work. 250 ·Inference & Serving·★ MEMBER·PAPER·9 min read Paper Walkthrough: The Personalization Mirage — LLMs Invent a Version of You, and Their Self-Reports Point the Wrong Way How much of what a memory-equipped LLM 'knows' about you was never actually said? MirageBench judges 143,616 claims across 150 personas and 6 tasks, finding that all 12 tested models over-infer 35–49% of their claims — and that the models reporting the least over-inference tend to fabricate the most. 251 ·Agents·★ MEMBER·PAPER·7 min read Paper Walkthrough: SWE-Bench ProMax — Measuring What Coding Agents Can Really Do with Large-Scale, Multilingual Refactoring A close read of the benchmark built to answer SWE-bench's saturation and grading defects: 170 expert-curated refactoring tasks across seven languages, averaging 11.4 modified files each. Why the best frontier model resolves only 41.2%, and why the dominant failure mode is refactoring that stops halfway. 252 ·Inference & Serving·★ MEMBER·PAPER·8 min read Speculative Decoding from Scratch — How a Tiny Draft Model Speeds Up an LLM Without Changing a Single Output Speculative decoding lets a small model draft several tokens ahead while a large model verifies them in one batch — provably identical outputs, 2–3x faster in practice. A from-scratch walkthrough of the mechanism and the intuition behind the acceptance rate α, straight from the original paper. 253 ·Device Physics·FREE·9 min read MOSFETs from the Ground Up — A Sluice Gate Opened by Voltage, and the Reality of Leakage The switch that fills every CPU and GPU, assuming no prior knowledge. From the sluice-gate analogy to threshold voltage, then the subthreshold leakage equation and why shrinking a transistor makes it leak more — the root of the power problem in one article. 254 ·Paper Deep-Dives·★ MEMBER·PAPER·9 min read Mixture of Experts (MoE) from Scratch — Routing and Load Balancing in the Switch Transformer A ground-up explanation of Mixture of Experts, the sparse architecture behind today's largest LLMs, built strictly from the Switch Transformer paper: the router math, expert capacity, the load-balancing loss, and the three tricks that make sparse training stable. 255 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: Macaron-V1 — A Frozen Base plus a Mixture of LoRAs, Built to Keep Learning After Launch A guided tour of Mind Lab's open agent-model family Macaron-V1, straight from the paper: the Mixture-of-LoRA architecture that stacks four specialist adapters on a frozen 744B base and picks one per turn, and the recursive self-improvement loop that trains the model and its harness as a versioned pair. 256 ·Model Families·★ MEMBER·9 min read The Llama Family from Scratch — The Main Line of Open LLMs How Meta's Llama became the main line of open LLMs. The contrarian bet behind Llama 1, the lineage running through Llama 3.x, the llama.cpp / GGUF / Ollama ecosystem that grew around it, and the licensing traps to read before you ship commercially — starting from no prior knowledge. 257 ·Model Families·★ MEMBER·PAPER·10 min read The Gemma Family from Scratch — Lineage, Inventions, and Where It Fits Google's open-weight LLM family, walked generation by generation. Distillation, local attention, effective parameters — the invention behind each release — plus the licensing detail that turns into an incident if you miss it, and how to actually run one on your own machine with Ollama. 258 ·Agents·★ MEMBER·PAPER·8 min read Paper Walkthrough: EnvACE — Agents That Rehearse the World Instead of Calling It EnvACE trains a tool-using LLM agent without ever calling an external environment: the same policy alternates between acting and playing the environment. We walk through role-wise GRPO, test-time private rehearsal, results on four agent benchmarks, and the limitations. 259 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: DAPD — Breaking the Teacher's "Cheat-Sheet Illusion" in Distillation with Dual Anchors When you distill from a teacher that can see the reference solution, the student learns to act as if invisible answers were still there — the privilege illusion. DAPD (arXiv:2608.01735) traces this failure to information asymmetry and fixes it with two levels of anchoring. A walkthrough grounded strictly in the paper itself. 260 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: CodeNib — A Multi-View Data System That Serves Repository Context to Coding Agents Coding agents grep their way through the same repository over and over. CodeNib (UC San Diego et al.) reframes this as a database problem — lexical, dense, and structural views over an immutable commit — and measures the whole lifecycle, caveats included. 261 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper explained: BDH-CQ — an AI that thinks without words. Recurrent memory plus latent reasoning resets ARC's cost frontier BDH-CQ (arXiv:2608.09888) never verbalizes its intermediate reasoning: demonstrations are written into a recurrent memory, and the answer is computed by iterating in a high-dimensional latent space. A 150M-parameter model reaches 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task, breaking the reported cost–accuracy Pareto frontier. We explain the paper using only its own text as evidence. 262 ·Paper Deep-Dives·★ MEMBER·PAPER·8 min read Paper Deep Dive: AskChem — Changing the Unit of Search from Papers to Provenance-Carrying Claims An NYU-led systems paper that swaps the retrieval unit of chemistry literature search from papers to claims grounded by a DOI and a verbatim quote. We walk through the 2.4M-claim index, evidence graph, and hybrid search design, the evaluation that took DOI resolution from 88.3% to 100%, and the paper's honest limitations. 263 ·Paper Deep-Dives·★ MEMBER·PAPER·8 min read Paper Walkthrough: Alpamayo — NVIDIA's Reasoning Model for Autonomous Driving A first-principles read of NVIDIA's driving VLA Alpamayo-R1: the Chain of Causation dataset that structures reasoning as cause and effect, flow-matching trajectory decoding that fits in 99 ms, and the three-stage training recipe that uses RL to punish gaps between what the model says and what it drives. 264 ·Inference & Serving·★ MEMBER·PAPER·9 min read Paper Deep-Dive: AgentOPSD — Finding the Turn That Won the Game with Recursive Bayesian Belief Updates When a long multi-turn episode only tells you 'success' or 'failure' at the very end, which turn actually mattered? AgentOPSD reinterprets self-distillation gaps as Bayesian evidence and recursively updates a success belief in log-odds space. We walk through the mechanism, the 89.1% ALFWorld result, the ablations, and the limitations — all from the paper itself. 265 ·Time Series·FREE·11 min read RNNs and LSTMs from Scratch — Why Learn Them in the Transformer Era Start from one idea — read a sequence one step at a time while carrying a state — then work out why multiplying the same matrix over and over kills the gradient, and what the three LSTM gates actually fixed. Ends with why Transformers took over, and where this recurrent idea still wins. 266 ·Large Language Models·FREE·PAPER·7 min read The Science of Prompt Engineering — What Is Proven and What Is Folklore Does telling a model to think step by step actually work? Working only from the Chain-of-Thought paper (Wei et al., 2022): what the experiments genuinely established — an ability that emerges only above a scale threshold — and where the line falls between that and the incantations everyone repeats. 267 ·Agents·★ MEMBER·PAPER·8 min read LLM Agents from Scratch — Designing the Tool-Use Loop Think, reach for a tool, look at what came back, think again. Working only from the text of the ReAct paper (Yao et al., 2022): why adding language to the action space is the whole idea, what function calling really implements, and the failure modes the authors actually measured — runaway loops and hallucinated tool calls. 268 ·Training & Alignment·★ MEMBER·PAPER·9 min read Instruction Tuning and RLHF from Scratch — How a Model Learns to Follow Orders A pretrained model ignores your instructions not because it lacks the ability but because it was optimised for something else. Working only from the InstructGPT paper (Ouyang et al., 2022): the three stages — SFT, reward model, RL — down to the equations, the claim that a 1.3B model beat a 175B one in human evaluation, and the limits the authors themselves put in writing. 269 ·Parallel & Distributed·★ MEMBER·10 min read Why GPUs Are Fast — The Execution Model and the Limits of Parallelism CPUs and GPUs do not mean the same thing by fast. Where the transistor budget goes, how SIMT bundles 32 threads into a warp, why branch divergence costs you, occupancy and register pressure — and finally Amdahl's law as a way to bound the payoff before you start, plus the profiler counters that tell you when the CPU is the bottleneck. 270 ·Parallel & Distributed·★ MEMBER·10 min read Distributed Training from Scratch — Data Parallel, Model Parallel, and When Communication Becomes the Bottleneck Why one machine is not enough, counted out in bytes; data parallelism and all-reduce; what ZeRO and FSDP actually shard; tensor and pipeline parallelism. Then the ratio of computation to communication that tells you where scaling stops paying — and gradient accumulation, NCCL settings and how to diagnose a hang. 271 ·VLMs & Multimodal·★ MEMBER·PAPER·7 min read Paper Deep Dive — CLIP: Putting Words and Images on One Map A reading of the CLIP paper (Radford et al., 2021) grounded strictly in its own text: how the in-batch positives and negatives work, why zero-shot classification is really a hypernetwork trick, the five points that prompt wording is worth, and the long list of limits the authors wrote themselves. 272 ·How Transformers Work·FREE·PAPER·10 min read Positional Encoding from Scratch — From Absolute Positions to RoPE A bare Transformer has no idea what word order is. Starting from why position information is needed at all, this article walks through sinusoidal absolute encodings, learned embeddings, and RoPE — the modern LLM standard — showing exactly why rotation encodes relative position. 273 ·Inference & Serving·★ MEMBER·8 min read LLM Quantization from Scratch — Why Losing Precision Doesn't Break It Cut a 16-bit weight down to 4 bits and the model keeps writing. Why doesn't it break? Starting from two numbers — scale and zero point — through the reason weights survive but activations resist (outliers), the fork between PTQ and QAT, what INT8 and INT4 actually do, and the single most common evaluation accident: signing off on perplexity alone. 274 ·Probability & Statistics·★ MEMBER·8 min read Probability and Statistics for AI — A Model's Output Is a Distribution Classifiers and language models do not return answers; they return probability distributions. Distributions, expectation, conditional probability and Bayes explained from the symbols up — building to the payoff: why maximum likelihood is where loss functions come from. Cross-entropy and MSE were derived, not invented. 275 ·Scaling & Power·★ MEMBER·9 min read The Physics of Power — Why Lowering Voltage Pays So Much Derive what one switching event costs, then read the dynamic power equation to see why voltage alone enters squared. From there: Dennard scaling and its end, the 60 mV/decade thermodynamic floor, dark silicon, and the resulting turn toward fixed-function circuits and low precision — ending with the arithmetic of what inference costs to run. 276 ·Accelerators·FREE·9 min read How Numbers Are Represented — From FP32 to FP8 and INT4 Open up the sign, exponent and mantissa fields and one rule falls out: the exponent buys range, the mantissa buys precision. From there — why bfloat16 was invented, why FP8 ships in two flavours, and what integer quantization actually destroys. Ends with which format to pick for training versus inference, and how to notice degradation before your users do. 277 ·Computer Architecture·★ MEMBER·9 min read The Memory Wall from Scratch — Why Moving Data Costs More Than Computing Multiplying two numbers is cheap; delivering them is not. Starting from the physics of charging a wire, we get to why DRAM latency never shrank, the orders of magnitude in the memory hierarchy, Little's law, machine balance and the roofline — and end with a procedure for deciding whether your kernel is compute bound or bandwidth bound. 278 ·Numerical Computing·★ MEMBER·8 min read The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes Why GEMM is everything: the anatomy of O(n³), memory bandwidth and arithmetic intensity, what actually makes a GPU fast, and the intuition behind tiling — ending with you able to estimate a model's training and inference FLOPs yourself. 279 ·Linear Algebra·FREE·9 min read Linear Algebra for AI — What Vectors and Matrices Are Actually Doing You will never compute a determinant to read an AI paper. What you do need is two ideas: a vector is a coordinate where meaning lives, and a matrix is a machine that carries coordinates somewhere else. From why a dot product means similarity to reading the attention equation symbol by symbol. 280 ·Inference & Serving·FREE·7 min read The KV Cache from Scratch — The Heart of Fast Inference An LLM emits one token at a time. Written naively, every single token costs a full recomputation of the whole sequence — a spectacular waste. The keys and values of past tokens never change again, and that one fact drops an entire order of magnitude. What you pay instead is memory, in an amount you can work out yourself, and that is why batch size and context length hit a ceiling. 281 ·Information Theory·★ MEMBER·9 min read Information Theory and AI — Where Cross-Entropy Loss Came From Information is surprise, entropy is average surprise, KL divergence is the gap between two distributions. Stack those three and the loss function you already use for classification falls out as the only reasonable answer — including why it involves a logarithm, and what perplexity is really counting. 282 ·Coding Theory·★ MEMBER·9 min read Entropy Coding from Scratch — From Huffman to Arithmetic Coding "Information content becomes code length" — that one line is all of compression. From Shannon's bound to building a Huffman code by hand, the integer-bit ceiling, the arithmetic coders and ANS that break it, and what is actually running inside JPEG, PNG, H.264 and zstd. Ending with how to diagnose a compression ratio that has stopped improving. 283 ·Search & Optimization·★ MEMBER·8 min read Dynamic Programming From Scratch — On Remembering Subproblems Why naive recursion explodes exponentially, what memoization and table-filling actually change, and Fibonacci, knapsack and edit distance taken apart in order — ending at the places edit distance shows up in real AI systems, from ASR word error rate to diffusion step schedules. 284 ·Data Structures·FREE·7 min read Choosing a Data Structure — Arrays, Hashes, Trees and Heaps What arrays, hash tables, trees and heaps each make fast, and what each one gives up in return. A pick-by-use-case table, plus which structures actually show up in tokenizers, vector search and KV caches. 285 ·Complexity·FREE·7 min read Complexity From Scratch — What Big-O Actually Measures What O(n), O(n log n) and O(n²) feel like as wall-clock time. Constant factors versus growth rate, trading time against space, and the three reasons your profiler disagrees with the textbook — assuming no prior knowledge. 286 ·Calculus & Optimization·FREE·7 min read Calculus for AI — The Gradient Is an Arrow Saying Which Way Is Better No epsilon-delta limits, no integration by parts. Training is measuring a slope and stepping the other way. A derivative is a multiplier, a gradient is a list of slopes, the chain rule is multiplication — and Jacobians and Hessians only need to be recognised, not computed. 287 ·Machine Learning Basics·FREE·6 min read What Machine Learning Really Is — Understanding “Learning” Without the Math What actually separates writing a program from training a model. Rules versus examples, memorization versus generalization, and a map of supervised, unsupervised, and reinforcement learning — with almost no math. 288 ·Machine Learning Basics·★ MEMBER·8 min read Overfitting and Evaluation Design — Be Suspicious of 99% Accuracy Why a model that fits its training data perfectly falls apart in production: the division of labour between train, validation and test, cross-validation, the data leakage that ruins more projects than any modelling mistake, and how accuracy lies on imbalanced data. 289 ·Deep Learning Basics·FREE·7 min read Neural Networks from Scratch — From One Neuron to Many Layers What a single neuron actually computes, and why stacking layers is pointless without an activation function — shown with a one-line proof that composing linear maps just gives you another linear map. Metaphor, math, an interactive figure, then fifteen lines of numpy. 290 ·Machine Learning Basics·FREE·8 min read Loss Functions and Optimization — How a Model Learns From Being Wrong Why MSE and cross-entropy have the shapes they do, what the gradient actually points at, and one step of gradient descent taken apart with equations, a draggable figure, and ten lines of numpy — divergence included. 291 ·Deep Learning Basics·★ MEMBER·9 min read Backpropagation from Scratch — It Is All Just the Chain Rule Why you can get gradients for ten million parameters for roughly the cost of one forward pass. The chain rule, computational graphs, a two-layer network worked by hand with real numbers, and where vanishing gradients come from — every symbol explained as it appears. 292 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Deep Dive — LoRA: Low-Rank Adaptation of Large Language Models: Why Low Rank Is Enough A re-reading of the LoRA paper (Hu et al., 2021) grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open. 293 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Deep Dive — Attention Is All You Need: What Dropping Recurrence Actually Proved A close reading of the Transformer paper grounded strictly in its own text: the scaled dot-product equation, why the square root of d_k is there, what the ablations exposed, and the limits the authors themselves flagged. 294 ·RAG & Retrieval·★ MEMBER·8 min read RAG Fundamentals and Design Patterns — Embeddings, Chunking, Reranking, and Evaluation from Scratch Retrieval-Augmented Generation explained from zero: the core intuition, chunking strategies, hybrid search and reranking, and the evaluation design that matters most. 295 ·How Transformers Work·FREE·8 min read Attention from Scratch — The Heart of the Transformer, Explained Visually Self-attention, the core mechanism behind ChatGPT, explained from zero: analogy, intuition, matrix mechanics, and runnable numpy code. No math background required.