#fine-tuning

23 articles

01 ·Training & Alignment·★ MEMBER·PAPER·7 min read Paper Deep Dive: Transformers Stop Thinking Too Early — How a One-Layer Rank-8 LoRA Stretches 3-Line Reference Following to 50 Pretrained LLMs follow in-context variable references for only two or three lines. Add a rank-8 LoRA (about 65K parameters) at one early layer with every weight frozen, and Qwen3-8B goes from 15.5% to 99% on 24-line chains. We walk through the mid-layer 'relay' behind it and the sharp limit on where the fix can go. 02 ·Training & Alignment·★ MEMBER·PAPER·7 min read Paper Deep Dive: Transformers Stop Thinking Too Early — How a One-Layer Rank-8 LoRA Stretches 3-Line Reference Following to 50 Pretrained LLMs follow in-context variable references for only two or three lines. Add a rank-8 LoRA (about 65K parameters) at one early layer with every weight frozen, and Qwen3-8B goes from 15.5% to 99% on 24-line chains. We walk through the mid-layer 'relay' behind it and the sharp limit on where the fix can go. 03 ·Training & Alignment·★ MEMBER·PAPER·7 min read Paper Explained: When Wording Changes an AI Reviewer's Score — RobustReview and SciCore A from-scratch walkthrough of a paper that defines 'rhetorical robustness' for AI reviewers whose scores shift when the same science is reworded. We cover 'false robustness', where stability alone fools you, and SciCore, which averages a full-manuscript judgment with a judgment of an extracted science core. 04 ·Training & Alignment·★ MEMBER·PAPER·8 min read Paper walkthrough: WanPE — turning a video prompt into a director's shooting plan WanPE rewrites a one-line request into a shot-by-shot plan covering camera, lighting and sound. We explain, from zero, why it builds training pairs backwards from real videos and how SC-GRPO penalizes dropped requirements. 05 ·Training & Alignment·★ MEMBER·PAPER·8 min read Paper walkthrough: WanPE — turning a video prompt into a director's shooting plan WanPE rewrites a one-line request into a shot-by-shot plan covering camera, lighting and sound. We explain, from zero, why it builds training pairs backwards from real videos and how SC-GRPO penalizes dropped requirements. 06 ·Training & Alignment·★ MEMBER·PAPER·10 min read Paper Walkthrough: SpeakerMem-R1 — A Two-Track Memory That Remembers Who Said What, About Whom, and When A from-scratch look at SpeakerMem-R1, a long-term memory for group conversations. It keeps verbatim messages alongside person- and group-level states, and trains a small note-taking model (Qwen2.5-3B) with reinforcement learning that scores memory owner by owner. 07 ·Training & Alignment·FREE·PAPER·19 min read QLoRA — Fine-Tuning a 65B Model on One GPU A 65-billion-parameter model in 16-bit weights is 130 GB before you have trained anything. QLoRA gets that onto a single 48 GB card by storing the frozen weights in a 4-bit format shaped like the weights themselves, quantizing the quantization constants, and paging the optimizer to CPU — while gradients still flow through those 4-bit weights into a small 16-bit adapter. 08 ·Training & Alignment·FREE·PAPER·17 min read LoRA from Scratch — Fine-Tuning a Giant by Training Two Thin Matrices Full fine-tuning of a 7B model needs over 100 GB of optimizer state for a job whose actual content is small. LoRA's claim is that the *change* a model needs is low-rank, so it can be stored as a product of two thin matrices, ΔW = BA. This article builds that from the cost problem, takes the equations apart one line at a time, explains why B starts at zero and what α/r is for, counts the parameters for a real model, and ends on the three decisions that actually matter: the rank, the target layers, and what merging does at inference time. 09 ·Training & Alignment·★ MEMBER·PAPER·14 min read Paper Explained: DataFlex-RL — Did 11 RLVR Data Policies Beat Uniform Sampling? Re-measured Over 12 Seeds "Train only on the good rollouts and the model gets stronger" is fast becoming RLVR folk wisdom. DataFlex-RL (arXiv:2609.06107) compares 13 selection, reweighting and mixture-adaptation configurations under one GRPO recipe and 12 seeds, and reports that every difference from uniform sampling has a 95% interval that crosses zero. We walk through the paper, including the finding that the choice of evaluation benchmarks reshuffles the winners. 10 ·Training & Alignment·FREE·PAPER·10 min read LoRA vs. Full Fine-Tuning — What You Give Up, What You Get LoRA squeezes a model's weight update into two thin matrices, ΔW = BA. This article starts from zero: why that is enough, what you actually trade away against full fine-tuning, how to pick the rank, and where merging quietly breaks. 11 ·RAG & Retrieval·FREE·8 min read RAG vs Fine-Tuning — Which One, and When The two main ways to make an LLM better, compared on four axes: knowledge freshness, cost, hallucination, and data requirements. From metaphor to math to interactive demos to the mistakes that break production systems. 12 ·Training & Alignment·★ MEMBER·PAPER·13 min read Paper Walkthrough: It Takes Two to Match — Co-Evolving Both Sides of Retrieval with RL A paper that rebuilds the first stage of search by having two LLMs write keywords — one for queries, one for items — and training them in alternation until their vocabularies meet. Explained from scratch. 13 ·Training & Alignment·★ MEMBER·PAPER·12 min read Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model. 14 ·Training & Alignment·★ MEMBER·PAPER·12 min read Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model. 15 ·Training & Alignment·★ MEMBER·PAPER·11 min read Paper Walkthrough: PaperGym — Turning One Paper Into a Graded Training Environment for Research Plans A research plan has no answer key, so reinforcement learning has no environment. This paper splits each paper into four drawers and builds the question and the grading criteria from different drawers, producing 20,000 training environments where paraphrasing the prompt no longer earns reward. 16 ·Training & Alignment·★ MEMBER·PAPER·10 min read PAWBench Explained — Can Video Generators Get the Odds Right, Not Just the Physics? If a video generator is a world model, it owes you more than one plausible rollout — it owes you the right distribution over futures. PAWBench measures that probabilistic alignment across 50 scenarios and 11 systems, and finds that no model gets all the requirements at once. 17 ·Training & Alignment·★ MEMBER·PAPER·9 min read Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep Learning Why a model that learns something new tends to lose what it already knew, explained from a single fact: weights are a shared resource. Walks through EWC, replay, and swappable LoRA adapters with equations and interactive figures, then explains why production teams still just mix the old data back in and retrain. 18 ·Inference & Serving·★ MEMBER·PAPER·15 min read Paper Explained: Agentic ESOpt — Drop Backprop, Jiggle the Weights, and Train Long-Horizon LLM Agents No gradients at all: spawn G perturbed copies of the model, run them in the environment, and nudge the weights toward the ones that scored well. That buys full-parameter updates of a 27B agent at inference-level memory (8.41GB) — and a 12.50-point lead over GRPO on 15-move Sudoku. A ground-up walkthrough of the NUS-led paper. 19 ·Training & Alignment·★ MEMBER·PAPER·11 min read Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself. 20 ·Training & Alignment·★ MEMBER·PAPER·11 min read Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself. 21 ·Training & Alignment·FREE·PAPER·9 min read Instruction Tuning and RLHF from Scratch — How a Model Learns to Follow Orders A pretrained model ignores your instructions not because it lacks the ability but because it was optimised for something else. Working only from the InstructGPT paper (Ouyang et al., 2022): the three stages — SFT, reward model, RL — down to the equations, the claim that a 1.3B model beat a 175B one in human evaluation, and the limits the authors themselves put in writing. 22 ·Paper Deep-Dives·FREE·PAPER·12 min read LoRA Paper Explained: Low-Rank Adaptation of Large Language Models (Hu et al., 2021) — Why Low Rank Is Enough The LoRA paper (Hu et al., 2021) explained, grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open. 23 ·Paper Deep-Dives·FREE·PAPER·12 min read LoRA Paper Explained: Low-Rank Adaptation of Large Language Models (Hu et al., 2021) — Why Low Rank Is Enough The LoRA paper (Hu et al., 2021) explained, grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open.