#fine-tuning
13 articles
01
·RAG & Retrieval·FREE·8 min read
RAG vs Fine-Tuning — Which One, and When
The two main ways to make an LLM better, compared on four axes: knowledge freshness, cost, hallucination, and data requirements. From metaphor to math to interactive demos to the mistakes that break production systems.
02
·Training & Alignment·★ MEMBER·PAPER·13 min read
Paper Walkthrough: It Takes Two to Match — Co-Evolving Both Sides of Retrieval with RL
A paper that rebuilds the first stage of search by having two LLMs write keywords — one for queries, one for items — and training them in alternation until their vocabularies meet. Explained from scratch.
03
·Training & Alignment·★ MEMBER·PAPER·12 min read
Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student
How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model.
04
·Training & Alignment·★ MEMBER·PAPER·12 min read
Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student
How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model.
05
·Training & Alignment·★ MEMBER·PAPER·10 min read
Paper Walkthrough: PaperGym — Turning One Paper Into a Graded Training Environment for Research Plans
A research plan has no answer key, so reinforcement learning has no environment. This paper splits each paper into four drawers and builds the question and the grading criteria from different drawers, producing 20,000 training environments where paraphrasing the prompt no longer earns reward.
06
·Training & Alignment·★ MEMBER·PAPER·10 min read
PAWBench Explained — Can Video Generators Get the Odds Right, Not Just the Physics?
If a video generator is a world model, it owes you more than one plausible rollout — it owes you the right distribution over futures. PAWBench measures that probabilistic alignment across 50 scenarios and 11 systems, and finds that no model gets all the requirements at once.
07
·Training & Alignment·★ MEMBER·PAPER·9 min read
Continual Learning and Catastrophic Forgetting — Why Models Can't Just Keep Learning
Why a model that learns something new tends to lose what it already knew, explained from a single fact: weights are a shared resource. Walks through EWC, replay, and swappable LoRA adapters with equations and interactive figures, then explains why production teams still just mix the old data back in and retrain.
08
·Inference & Serving·★ MEMBER·PAPER·15 min read
Paper Explained: Agentic ESOpt — Drop Backprop, Jiggle the Weights, and Train Long-Horizon LLM Agents
No gradients at all: spawn G perturbed copies of the model, run them in the environment, and nudge the weights toward the ones that scored well. That buys full-parameter updates of a 27B agent at inference-level memory (8.41GB) — and a 12.50-point lead over GRPO on 15-move Sudoku. A ground-up walkthrough of the NUS-led paper.
09
·Training & Alignment·★ MEMBER·PAPER·11 min read
Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer
When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself.
10
·Training & Alignment·★ MEMBER·PAPER·11 min read
Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer
When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself.
11
·Training & Alignment·★ MEMBER·PAPER·9 min read
Instruction Tuning and RLHF from Scratch — How a Model Learns to Follow Orders
A pretrained model ignores your instructions not because it lacks the ability but because it was optimised for something else. Working only from the InstructGPT paper (Ouyang et al., 2022): the three stages — SFT, reward model, RL — down to the equations, the claim that a 1.3B model beat a 175B one in human evaluation, and the limits the authors themselves put in writing.
12
·Paper Deep-Dives·★ MEMBER·PAPER·12 min read
Paper Deep Dive — LoRA: Low-Rank Adaptation of Large Language Models: Why Low Rank Is Enough
A re-reading of the LoRA paper (Hu et al., 2021) grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open.
13
·Paper Deep-Dives·★ MEMBER·PAPER·12 min read
Paper Deep Dive — LoRA: Low-Rank Adaptation of Large Language Models: Why Low Rank Is Enough
A re-reading of the LoRA paper (Hu et al., 2021) grounded strictly in its own text: what BA actually means, the 21x amplification factor the authors measured, why r=1 sufficed, and the questions they left open.