#self-distillation
3 articles
01
·Inference & Serving·★ MEMBER·PAPER·12 min read
Paper Walkthrough: DART-SD — Training Tool-Calling Agents Without Flattening the Diamond
When a multi-turn tool-calling task has order-independent sub-goals, the set of correct solutions spreads out into a diamond lattice. This walkthrough explains why whole-trajectory imitation crushes that structure, and how locating the first point where a rollout leaves recoverable territory — and supervising only what comes after it — changes the picture.
02
·Distillation & Compression·★ MEMBER·PAPER·9 min read
A Field Guide to Distillation Recipes — logit, feature, attention, self
Every distillation recipe is an answer to one question: which part of the teacher should the student match? This guide lines up output (logit), intermediate feature (FitNet), attention, and self-distillation in one table and four equations, then turns the choice into a decision you make from constraints — is the teacher behind an API, do the layers line up, do the head counts match.
03
·Inference & Serving·★ MEMBER·PAPER·9 min read
Paper Deep-Dive: AgentOPSD — Finding the Turn That Won the Game with Recursive Bayesian Belief Updates
When a long multi-turn episode only tells you 'success' or 'failure' at the very end, which turn actually mattered? AgentOPSD reinterprets self-distillation gaps as Bayesian evidence and recursively updates a success belief in log-odds space. We walk through the mechanism, the 89.1% ALFWorld result, the ablations, and the limitations — all from the paper itself.