Paper Review: S²VOPD — Don't Strengthen the Teacher, Blur the Student's View

A walkthrough of S²VOPD, which improves vision-language models without labels, rewards, or a stronger teacher. The trick is inverting where the asymmetry comes from: degrade the student's view instead of privileging the teacher's.

Self-Supervised Visual On-Policy Distillation

Is There Any Way Up That Isn't "Get a Stronger Teacher"?

Distillation copies the answering behaviour of a capable model (the teacher) into another model (the student). On-policy distillation pushes this one step further. Instead of having the student transcribe a model answer, you let the student write in its own words, and at every fork in that text the teacher marks it up: "here is the probability I would assign to each next word." Because the grading happens where the student actually stands, the gap between practice and performance narrows.

But there is a precondition: the teacher has to know something the student doesn't. The paper calls this the teacher–student asymmetry and places it at the centre of distillation (§1). If the one holding the red pen knows nothing the student doesn't, all that gets written down is self-affirmation. Conventionally this asymmetry has come from one of two places: making the teacher a larger, stronger model, or feeding the teacher privileged information — reference solutions, environmental feedback, ground-truth regions of interest. Both must be supplied from outside, and the stronger models get, the harder it becomes for human supervision to keep up.

So the paper asks: when nothing privileged is available, where can informative asymmetry come from? The answer was to invert its direction. Rather than adding information to the teacher, subtract information from the student. The teacher sees the original image; the student sees a degraded version of the same image. That alone recreates the situation where the teacher holds something the student cannot see — for free.

The Analogy: The Same Photo Through Frosted Glass

Two people are looking at the same photograph. The teacher looks through clear glass, the student through frosted glass. Both answer the same question: "how many people are sitting on that bench in the back?" When the student says "two, maybe" and the teacher says "three," the disagreement did not come from a difference in ability. It came from a difference in how much was visible. That is why the disagreement carries a direction — look harder — even though nobody in the room knows the right answer. This is the core of S²VOPD (Self-Supervised Visual On-Policy Distillation).

The paper targets fine-grained perception. Evaluation uses six perception benchmarks — V*Bench, ZoomBench, HR-Bench 4K and 8K, MME-RealWorld and its Chinese subset — plus three math reasoning benchmarks: MathVista, MathVerse and MathVision (§4.1). These are dominated by questions where the answer hinges on one small spot in a high-resolution photo, which is exactly why artificially removing visible information becomes a meaningful learning signal. Training uses image–question pairs drawn from the natural-image domain of FineVision, with no answer labels at all (§4.1).

The Mechanism: The Teacher Is the Student's Own Shadow

Write the student as $\pi_\theta$ and the teacher as $\pi_\phi$. The teacher is not a separately trained stronger model — it is an exponential moving average (EMA) of the student's own weights (§3.1).

Here $\phi$ is the teacher's weights, $\theta$ the student's, and $\eta$ the update rate. Equation (1) says only "nudge the teacher's weights a little toward the student's," so the teacher is a smoothed version of the student's slightly earlier self — a shadow. Stated in words: the teacher is a running average of the selves the student has already been, and the smaller $\eta$ gets, the more sluggishly it moves and the less it is dragged around by the student's most recent wobbles.

For each image–question pair $(x,q)$ the method draws a stochastic transformation $T$ and builds the student's view $\tilde{x}=T(x)$. The student writes its own responses while looking at $\tilde{x}$, and the teacher grades those very same partial sentences while conditioned on the original image $x$. Augmentation is applied to the student side only; the teacher always sees the original (§3.1).

Letting the student write is what makes this on-policy. Comparing teacher and student on a pre-written reference answer means grading on text the student would never actually produce, so the distribution of text seen during training drifts from the one seen at inference. Have the student write $n$ of its own responses (8 by default) and grade those, and the red pen lands only on the forks where the student genuinely wavers. No gradient flows into the teacher; it moves only through the EMA update above, which is why the paper can call this a self-supervised setup rather than a two-model one.

The Loss: Two Distributions Compared on the Same Sentence

The objective measures, at each token position along a trajectory the student wrote itself, the divergence between teacher and student distributions, and minimises the average (§3.1).

$y$ is the student's response, $y_{<t}$ is that response up to position $t$, and $D$ measures the gap between two distributions. Show both models the same partial sentence, line up their next-token distributions, sum, and divide by the response length. The point is that the only thing differing between teacher and student is the image they are conditioned on. Question and context are shared, so the gap traces back purely to what was visible.

Put in words, the objective says: replay the sentence the student wrote, one token at a time, and at every point ask how differently the same model would have continued had it been looking at the clean image — then shrink that disagreement on average. Dividing by the response length keeps long answers from dominating the loss simply by having more token positions in them.

For $D$ the paper uses the generalized Jensen–Shannon divergence (JSD). With the mixture $m_t=\alpha\,\pi_\phi+(1-\alpha)\,\pi_\theta$,

which says: blend the two distributions into a halfway mixture $m_t$, then add up how far each one sits from that middle, weighted by $\alpha$. Rather than measuring one distribution against the other as the reference, the yardstick is planted between them.

$\alpha$ is the mixture weight, $0.5$ by default. At $0$ it leans toward forward KL, at $1$ toward reverse KL, and $0.5$ sits between them (for KL itself, see KL Divergence from Scratch). In practice both distributions are restricted to the teacher's top-k tokens and renormalised over that support, which keeps the objective from being dragged around by the long tail of the vocabulary (§3.1).

for x, q in loader:
    xs = add_noise(downscale(x, s=uniform(0.3, 0.6)), p=0.5)  # degrade student only
    ys = student.sample(xs, q, n=8)                           # student writes its own
    for y in ys:
        p_t = teacher.dists(x,  q, y)      # teacher scores same prefix, clean image
        p_s = student.dists(xs, q, y)      # student, degraded image
        p_t, p_s = topk_renorm(p_t, p_s, k)
        step(js_divergence(p_t, p_s, alpha=0.5).mean())
    ema_update(teacher, student, eta=0.05)

How to Degrade: Four Families and the Best Recipe

The paper organises operators by how they change the visual information available to the student, giving four families (§3.2). Information reduction — downscaling, blur, pixelation, spectral band-stop, Gaussian noise, visual-token dropping — cuts usable information while preserving spatial layout. Geometric operators — rotation, translation, cropping, zoom-out with padding — change spatial organisation and, unlike the others, can change the answer itself for spatially grounded questions. Photometric operators (brightness, contrast, saturation, hue) alter appearance only, and occlusion (random erasing, GridMask) removes localised regions.

The strongest configuration stacked two operators from the information-reduction family (§3.2).

Read in words, equation (4) is two moves and nothing clever: shrink the image, then sprinkle noise over what's left. $A_{\mathrm{down}}$ is downscaling and $A_{\mathrm{noise}}$ additive Gaussian noise. The scale $s$ is drawn fresh from $0.3$–$0.6$ each time, and since the global augmentation probability is $1$, the student's view is always degraded. Noise is applied with probability $0.5$, following a DDPM forward step at $t{=}200$ (additive noise of standard deviation roughly $0.11$). One detail carries weight: the image is not resized back after downscaling. Because it isn't, the number of visual tokens reaching the student actually drops, so information falls and rollout plus forward-pass cost fall with it (on visual tokens, see How VLMs Came Together).

Three Findings

The numbers below were measured under a 2,048-token budget due to limited compute, so they are internally comparable rather than absolute (§4.3).

1. The asymmetry itself is what works. The base model scores $70.58\%$. Symmetric self-distillation, with $T$ set to the identity, reaches only $65.21\%$ — worse than where it started. Applying each family alone gives information reduction $75.65\%$, photometric $74.40\%$, geometric $74.30\%$, occlusion $72.44\%$: all four beat both the base model and symmetric self-distillation by a wide margin.

2. Strength has a sweet spot. Across every family, performance rises and then falls. Resolution reduction peaks at $75.65\%$ for the $0.3$–$0.6$ range, against $75.25\%$ weaker and $75.05\%$ stronger. Blur traces the same arc, $74.20\% \to 75.71\% \to 75.07\%$. Instead of the strength parameter, the paper plots accuracy against the token-level JS divergence measured over the first ten training steps, and finds a consistent turn from rising to falling at roughly $0.014$.

3. A bigger gap is not a better gap. Cropping exposes the limit. It degrades monotonically with strength — $71.53\% \to 68.76\% \to 67.44\%$ — and the strongest crop produces the largest gap in the entire study while barely clearing symmetric self-distillation. The reason is plain: cropping can delete the very evidence the question depends on. A gap that arises because the input became unanswerable is large but empty.

How Far It Got

Trained on 12K natural-image questions from FineVision, Qwen3.5-4B goes from $70.68\%$ to $77.44\%$ average accuracy across the six perception benchmarks, a gain of $6.76\%$ (§4.2). That 4B model beats every open-source model in the table — including Qwen3-VL-Instruct-235B at $75.75\%$ — along with the GPT-5 series, and matches Qwen3.5-397B. It does this while surpassing the privileged-information methods ZwZ, Vision-OPD and Opsd, using no ground-truth regions or answers.

The comparison table also exposes an asymmetry in what each method is good at. Methods using privileged visual information lift perception but hurt math reasoning: Opsd drops MathVision by $9.3\%$ at 4B, and ZwZ drops MathVerse by $27.1\%$ at 4B. Self-rewarding RL runs the other way, improving math while gaining little on perception. When training data is held fixed (Vision-OPD-6K, 65 steps), S²VOPD posts the best 4B nine-benchmark average at $75.33\%$ against Vision-OPD's $74.77\%$ — precisely because it improves both at once (§4.2).

What Was Actually Doing the Work

Show the student the same clean image as the teacher, removing only the asymmetry, and the average falls to $70.52\%$ against the base model's $70.68\%$ — the entire gain disappears. With no information gap, the paper notes, the objective amplifies the teacher's confident errors instead of correcting the student's. Keep the augmentation but freeze the teacher at the base model and you still get $75.95\%$, just $0.40\%$ below the default, retaining $93\%$ of the gain. The EMA is a stable implementation choice; the supervision comes from the constructed asymmetry (§4.4).

The divergence comparison lands the same way: forward KL $74.74\%$ < reverse KL $75.49\%$ < JSD $76.05\%$. Forward KL is too coverage-seeking, forcing the student to match detail its degraded view cannot reach. Reverse KL is too mode-seeking, letting the student pile onto high-confidence peaks and discard softer corrective signal. JSD sits in between (§4.4).

Limits and What Remains Unverified

Only Qwen3.5 at 4B and 9B were used (§4.1); reproduction on other model families or scales is outside this paper. Training data is natural images, so the recipe is untested on documents and charts, where downscaling and noise can make text unreadable. The §4.3 analysis runs on a 2,048-token budget, so its numbers sit below the main results and are only valid internally, as the paper itself states. The comparison against self-rewarding RL is conservative in the baselines' favour: TTRL, Intuitor and RENT collapse during training, so their best of 13 checkpoints is reported, while S²VOPD is reported from its final checkpoint (§4.2). And there is no theory behind either the inverted-U shape or the $0.014$ JSD threshold — they are observed regularities.

One reading I'd add: because the teacher is an EMA of the student, what this method can surface is capability the student already has but fails to produce from a degraded input. It is not a way to inject knowledge the model lacks. The paper doesn't state this, so I flag it as my interpretation rather than theirs.

Using This in Practice

Who, and when. An ML engineer trying to improve a small VLM on in-house imagery, in the situation where image–question pairs exist but answer labels and region annotations do not. Since it amounts to adding a few lines of image transformation on the student side, it's worth trying before you go price out an annotation contract.

Parameters you'll touch (§3.2, §4.1): downscale factor $s\sim\mathcal{U}(0.3,0.6)$ and do not resize back; Gaussian noise at probability $0.5$, standard deviation about $0.11$; global augmentation probability $p=1$; JSD with $\alpha=0.5$ over the teacher's top-k support; EMA rate $\eta=0.05$; $n=8$ rollouts per prompt, batch of 96 prompts, learning rate $2\times10^{-6}$, 10 warmup steps, 65–130 optimizer steps; vLLM as the rollout engine.

What to monitor. Rather than groping for the right strength, measure token-level JS divergence over the first ten steps and check that it lands near $0.014$ (§4.3). It's a single yardstick that stays comparable when you swap augmentation types.

Ways this bites you.

1. "Let's just self-distill" backfires. Symmetric self-distillation without augmentation scored below the base model ($65.21\%$ vs $70.58\%$). If a bug means the augmentation never reaches the student, performance quietly degrades.
2. Don't tune to maximise the gap. The strongest crop produced the biggest gap and the worst score. Watch whether the gap is near $0.014$, not whether it's large.
3. Geometric transforms can change the answer. Rotate or translate and "what's on the left of the frame" has a different correct answer (§3.2). If spatially grounded questions are in your mix, skip that family.
4. Don't resize back after downscaling. If you do, visual token count doesn't drop, and both the cost saving and the character of the information reduction change. Most resize helpers restore the original size by default, so check this explicitly.
5. Don't fall back on forward KL. It came last at $74.74\%$, because it makes the student imitate detail it cannot see.

The reasoning you'll be asked for in a design review. "If teacher and student are the same model, where does the learning signal come from?" It comes from a difference in conditioning, not capability. Because the comparison happens on the same partial sentence, the difference traces purely to what was visible — and it points in the direction of look harder.

Takeaways

• Distillation needs the teacher's advantage, not the teacher's strength. Subtract information from the student and you manufacture the same advantage for free
• Two conditions govern whether it works: the gap must be sufficient but not excessive (JSD around $0.014$), and the gap must not destroy the answer to the question
• Downscaling plus Gaussian noise is the most robust default, and since the student's input shrinks, training cost goes down rather than up

For a different route to distillation without labels or a stronger teacher, u-OPSD and its "self-majority-vote" is a useful companion read on how non-privileged learning signals can be constructed.