Paper Explained: Beyond Teacher Assignment — Why Choosing the Right Teacher Isn't Enough When You Distill Many Specialists into One Model

When MOPD distills math, code and instruction-following specialists into one student, the instruction-following teacher shouts so loudly that math barely transfers. DN-MOPD measures each teacher's feedback spread and evens out the volume, with a one-line change.

Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation

What this paper claims

The paper is "Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation" (arXiv:2609.35347, September 2026).

Here is the abstract in plain terms. Reinforcement learning (RL) can turn one language model into several specialists, each strong at a single skill such as math, coding or instruction following (IF). Users, though, want one model that does all of it. Multi-teacher on-policy distillation (MOPD) merges the specialists by having them teach one student: the student answers each prompt, and the specialist for that prompt's domain gives feedback on every token. That routing decides who teaches, but not how hard the feedback pushes the shared student. Across three sizes of Qwen3.5, the authors find that the MOPD student doesn't beat a student taught by the best single specialist and keeps little of the math specialist's edge. The feedback is lopsided: IF feedback is several times more spread out than math feedback and dominates the student's updates. Their method, DN-MOPD, keeps the routing and rescales each domain's feedback by its measured spread. On six public benchmarks it raises the average score over MOPD at every size, across three seeds and two answer-length limits, and recovers most of the lost math gain.

An analogy: three tutors at different volumes

Picture one student with three tutors: math, programming and writing. Each tutor marks the problems in their own subject. So far, so fair.

The writing tutor, however, yells at every single word — "brilliant here!", "terrible here!" The math tutor mutters corrections under their breath. The student only has one head, so day after day the loud voice wins and the math corrections barely register.

The paper's point is that the problem was never who marks the work. It was the volume.

Background: on-policy distillation and MOPD

Knowledge distillation copies a large teacher's behavior into a student. On-policy distillation (OPD) is the variant where, instead of copying the teacher's model answers, the student writes its own answer and the teacher grades it token by token.

MOPD extends this to several teachers. Each prompt carries a domain label — math, code or IF — and the specialist for that label does the grading. The paper calls this Label routing. According to the authors, MOPD has been used in the post-training of several recent frontier models (§1).

What went wrong

For each of Qwen3.5-9B, 4B and 2B, the authors trained math, code and IF specialists independently with RL from the same starting model, then merged them with MOPD (§3). The results were disappointing:

• At no size did the MOPD student beat the student taught by the strongest single teacher.
• With an 8K-token evaluation cap it kept only 14–32% of the math specialist's gain, and at 16K it showed no gain over its initialization (§1).

Digging in, they found the teacher-minus-student log-probability gap (the $A_t$ we'll define shortly) had very different spreads per domain. In the first training batch, IF values were 2.3–4.4 times as dispersed as the pooled signal, while math was about half as dispersed (§1, §2). For the initial 4B student under equal weights, the IF loss supplied 94% of the combined gradient (§1).

When several signals fight over the same parameters and one of them is too strong, it washes the others out. It's the same family of problem as gradient descent with a learning rate that's too high, where each step overshoots the valley. Try it below.