Students Professional
JA EN

#on-policy

3 articles

01 ·Distillation & Compression·★ MEMBER·PAPER·9 min read On-Policy Distillation — Learning From What the Student Actually Writes Classic distillation has the student copy sentences the teacher wrote. On-policy distillation has the student write, then lets the teacher mark it up. The difference is one symbol in the loss — and that symbol removes exposure bias, turns distillation into a form of RL, and opens the door to self-distillation methods like u-OPSD and AgentOPSD. 02 ·★ MEMBER·PAPER·12 min read Paper Review: S²VOPD — Don't Strengthen the Teacher, Blur the Student's View A walkthrough of S²VOPD, which improves vision-language models without labels, rewards, or a stronger teacher. The trick is inverting where the asymmetry comes from: degrade the student's view instead of privileging the teacher's. 03 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Walkthrough: DAPD — Breaking the Teacher's "Cheat-Sheet Illusion" in Distillation with Dual Anchors When you distill from a teacher that can see the reference solution, the student learns to act as if invisible answers were still there — the privilege illusion. DAPD (arXiv:2608.01735) traces this failure to information asymmetry and fixes it with two levels of anchoring. A walkthrough grounded strictly in the paper itself.