Students Professional
JA EN

#post-training

10 articles

01 ·Agents·★ MEMBER·PAPER·12 min read Paper Walkthrough: Terminal-Universe — Turning Agent Logs Back Into Reusable Execution Environments Replay the file operations recorded in an agent trajectory, have a completion agent fill in what's missing, and you get an executable workspace back. The pipeline yields 37.3k environments and lifts Qwen3.5-27B by 11.9 points on Terminal-Bench 2.1. 02 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: One Training Example Keeps On-Policy Distillation Improving for Hundreds of Steps Trained on a single query, on-policy distillation still improves for hundreds of steps and recovers most of full-data OPD's gain. The paper explains this with state coverage and absorption rate, and concludes OPD is data-overfed but algorithm-starved. 03 ·Agents·★ MEMBER·PAPER·14 min read Paper Walkthrough: Aspire — Can Models Self-Evolve from Vague Goals? Hand an agent nothing but "get better at mathematical reasoning" and let it decide what to learn and how to check itself. Aspire measures the result on 520 hidden, expert-written items — and finds that agents close the training loop far more reliably than the capability loop. 04 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: From Production Traffic to Post-Training — Folding 200 Internal Apps Into One Self-Hosted LLM How a company forced to self-host by data-residency rules collapsed a sprawling model fleet into one checkpoint: classify production failures by hand, train one GRPO expert per weak axis, then merge the weights with SLERP. Includes three real reward-hacking failures. 05 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Explained: Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement A third to a half of the teacher's token-level grades are wrong, yet the student improves just as fast either way. This paper traces on-policy distillation's gains not to imitating a teacher but to suppressing the student's own low-probability tokens — and drops the teacher entirely. 06 ·★ MEMBER·PAPER·9 min read Paper Walkthrough: On-Policy Self-Distillation in Diffusion Models — Turning Reward into a Target You Can Aim At When you train an image generator with rewards, a score on the finished picture never tells the model how to change its intermediate denoising predictions. DiffusionOPSD builds explicit targets from reward gradients and fits them using an EMA copy of the model itself. A ground-up explanation, strictly within what the abstract states. 07 ·★ MEMBER·PAPER·9 min read Paper Walkthrough: Annotations as Rollouts — Dropping the Ground Truth Into the Group as a Ninth Answer RL post-training for video MLLMs starves because sampled rollouts almost never contain the right answer. OraRL adds the annotation itself to the group as an extra rollout, then repairs the advantage inversion that naive mixing causes. 08 ·Large Language Models·★ MEMBER·PAPER·11 min read Paper Walkthrough: SA-MRPO — Stop Studying the Subject You've Already Aced When you train with several reward objectives at once, gradient budget keeps flowing to objectives that are already solved. SA-MRPO (arXiv:2608.16072) measures how saturated each objective is and discounts its weight accordingly, redirecting optimization toward whatever headroom remains. A walkthrough grounded strictly in the paper. 09 ·Inference & Serving·★ MEMBER·PAPER·14 min read Paper Walkthrough: No Gold Answers, No Stronger Teacher — How u-OPSD Distills From Its Own Majority Vote u-OPSD (arXiv:2608.06296) replaces the gold solution in on-policy self-distillation with the model's own majority vote, then corrects only the rollouts that disagree with it. A from-scratch walkthrough grounded solely in the paper. 10 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: DAPD — Breaking the Teacher's "Cheat-Sheet Illusion" in Distillation with Dual Anchors When you distill from a teacher that can see the reference solution, the student learns to act as if invisible answers were still there — the privilege illusion. DAPD (arXiv:2608.01735) traces this failure to information asymmetry and fixes it with two levels of anchoring. A walkthrough grounded strictly in the paper itself.