#post-training

20 articles

01 ·Time Series·★ MEMBER·PAPER·7 min read Paper Explained: Video Generation Models: A Survey of Post-Training and Alignment — Four Toolboxes for Making Video Models Do What You Ask A from-scratch reading of the survey that organizes how pretrained video generators are steered toward instruction-following, temporal coherence, physical plausibility, and safety without retraining—via supervised fine-tuning, self-training and distillation, preference and reward optimization, and inference-time methods. 02 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Explained: Beyond Teacher Assignment — Why Choosing the Right Teacher Isn't Enough When You Distill Many Specialists into One Model When MOPD distills math, code and instruction-following specialists into one student, the instruction-following teacher shouts so loudly that math barely transfers. DN-MOPD measures each teacher's feedback spread and evens out the volume, with a one-line change. 03 ·Inference & Serving·★ MEMBER·PAPER·9 min read Paper Explained: Post-Training Leaves Behavioral Shadows on Unrelated Decisions — capability transfer through one unrelated word Ask a code-tuned model to pick between 'jacket' and 'tie' a few thousand times, and another model can inherit part of its coding gain. A from-scratch walkthrough of Active Taskless Distillation (ATD): the math, the controls, and the limits. 04 ·Inference & Serving·★ MEMBER·PAPER·7 min read Paper Explained: When EOS Tokens Disagree — Why On-Policy Distillation Students Forget How to Stop In on-policy distillation, student responses can grow until they hit the generation budget. This paper (arXiv:2609.20511) traces a major cause to student and teacher marking 'the end' with different special tokens. We walk through it from scratch, using only the paper itself. 05 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Explained: Negative Self-Distillation — Learning to Reason by Steering Away From Your Worst Self Self-distillation from ground-truth solutions (OPSD) can strip a reasoning model of its habit of pausing and double-checking. NSD has the model play a 'careless reasoner' and trains only to move away from that bad teacher — no labels required. A from-scratch walkthrough. 06 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper Walkthrough — Feyospace-v1: Seven People, 164k Trajectories, and a Cyber Model A seven-person team pushed open-weight models to leading agentic cyber capability — not by scaling, but by building resettable executable environments and auditing 164,269 trajectories. A ground-up walkthrough of the design and its stated limits. 07 ·Training & Alignment·★ MEMBER·PAPER·14 min read Paper Explained: DataFlex-RL — Did 11 RLVR Data Policies Beat Uniform Sampling? Re-measured Over 12 Seeds "Train only on the good rollouts and the model gets stronger" is fast becoming RLVR folk wisdom. DataFlex-RL (arXiv:2609.06107) compares 13 selection, reweighting and mixture-adaptation configurations under one GRPO recipe and 12 seeds, and reports that every difference from uniform sampling has a 95% interval that crosses zero. We walk through the paper, including the finding that the choice of evaluation benchmarks reshuffles the winners. 08 ·Inference & Serving·★ MEMBER·PAPER·10 min read Paper Walkthrough: Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training — training the draft alongside a 122B policy at 256K tokens Rollout generation dominates RL post-training time. NVIDIA speeds it up with speculative decoding and keeps training the draft model alongside the policy — and shows how to get past the two walls that context parallelism and pipeline parallelism put in the way. A from-scratch walkthrough grounded in the paper text. 09 ·★ MEMBER·PAPER·17 min read Paper Walkthrough: Miles v0.1 — Anatomy of a Post-Training Stack Built to Survive Production A read-through of Miles, a full-stack system for agentic RL post-training of frontier MoE models: how it hunts down the numerical drift between the rollout engine and the trainer, and what a 744B model actually measured on 64 GPUs. 10 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: On-Policy Reverse Distillation — Borrowing a Weak Teacher's Direction, Not Its Answers OPRD takes the policy change a weak teacher acquired during post-training and uses it only as a direction to amplify the student's own policy gradient — never as an optimization target. A ground-up walkthrough. 11 ·Agents·★ MEMBER·PAPER·11 min read Paper walkthrough: Knowing When Not to Reuse — conditional experience transfer in autonomous post-training (BCIT) Deciding, before you spend a single GPU-hour, whether an update that worked before should be applied to the parent model you have now. BCIT scores source strength, current applicability and named conflicts, then picks Reject, Validate or Train. 12 ·Agents·★ MEMBER·PAPER·13 min read Paper Walkthrough: Terminal-Universe — Turning Agent Logs Back Into Reusable Execution Environments Replay the file operations recorded in an agent trajectory, have a completion agent fill in what's missing, and you get an executable workspace back. The pipeline yields 37.3k environments and lifts Qwen3.5-27B by 11.9 points on Terminal-Bench 2.1. 13 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: One Training Example Keeps On-Policy Distillation Improving for Hundreds of Steps Trained on a single query, on-policy distillation still improves for hundreds of steps and recovers most of full-data OPD's gain. The paper explains this with state coverage and absorption rate, and concludes OPD is data-overfed but algorithm-starved. 14 ·Agents·★ MEMBER·PAPER·14 min read Paper Walkthrough: Aspire — Can Models Self-Evolve from Vague Goals? Hand an agent nothing but "get better at mathematical reasoning" and let it decide what to learn and how to check itself. Aspire measures the result on 520 hidden, expert-written items — and finds that agents close the training loop far more reliably than the capability loop. 15 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: From Production Traffic to Post-Training — Folding 200 Internal Apps Into One Self-Hosted LLM How a company forced to self-host by data-residency rules collapsed a sprawling model fleet into one checkpoint: classify production failures by hand, train one GRPO expert per weak axis, then merge the weights with SLERP. Includes three real reward-hacking failures. 16 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Explained: Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement A third to a half of the teacher's token-level grades are wrong, yet the student improves just as fast either way. This paper traces on-policy distillation's gains not to imitating a teacher but to suppressing the student's own low-probability tokens — and drops the teacher entirely. 17 ·FREE·PAPER·9 min read Paper Walkthrough: Annotations as Rollouts — Dropping the Ground Truth Into the Group as a Ninth Answer RL post-training for video MLLMs starves because sampled rollouts almost never contain the right answer. OraRL adds the annotation itself to the group as an extra rollout, then repairs the advantage inversion that naive mixing causes. 18 ·Large Language Models·★ MEMBER·PAPER·11 min read Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper. 19 ·Inference & Serving·FREE·PAPER·14 min read Paper Walkthrough: No Gold Answers, No Stronger Teacher — How u-OPSD Distills From Its Own Majority Vote u-OPSD (arXiv:2608.06296) replaces the gold solution in on-policy self-distillation with the model's own majority vote, then corrects only the rollouts that disagree with it. A from-scratch walkthrough grounded solely in the paper. 20 ·Inference & Serving·FREE·PAPER·8 min read Paper Walkthrough: DAPD — Breaking the Teacher's "Cheat-Sheet Illusion" in Distillation with Dual Anchors When you distill from a teacher that can see the reference solution, the student learns to act as if invisible answers were still there — the privilege illusion. DAPD (arXiv:2608.01735) traces this failure to information asymmetry and fixes it with two levels of anchoring. A walkthrough grounded strictly in the paper itself.