Summary (Overview)

• Identifies a key limitation in multimodal on-policy distillation (OPD): teacher corrections are "source-mixed," combining visual signals with linguistic priors and teacher-specific effects, making it unclear which corrections are visually supported.
• Introduces Visual Attribution Distillation (VAD), a counterfactual target-reconstruction algorithm that estimates the visually attributable part of a teacher correction by contrasting evidence-present and evidence-removed teacher views.
• Develops a signed proxy $u_t$ for the visual evidence direction, projecting the original teacher correction onto this proxy to obtain an intervention-aligned component and a residual, then reconstructing a student-anchored target.
• Achieves state-of-the-art results across six fine-grained visual benchmarks at both 4B and 9B model scales, outperforming direct privileged-view distillation and visual-advantage weighting methods.
• Demonstrates semantic separation: the attributed component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer.

Introduction and Theoretical Foundation

Multimodal large language models (MLLMs) often fail due to small perceptual misses—overlooking a word, confusing an attribute, or misreading a spatial relation—while maintaining fluent but visually ungrounded responses. Knowledge distillation transfers predictive structure from a stronger teacher model, and on-policy distillation (OPD) makes this supervision relevant to deployment by querying the teacher on prefixes generated by the student itself.

The paper identifies two key limitations of existing multimodal OPD approaches:

1. Vision-OPD (Yuan et al., 2026) conditions the teacher on an evidence-centered crop but directly distills its complete next-token distribution. Diagnostic analysis shows that only 23.2% of the strongest response-token corrections for the 4B model and 22.8% for the 9B model are strongly aligned with the teacher's evidence-conditioned response; the remaining corrections are "source-mixed."

2. VA-OPD (Liu et al., 2026) and V-Zero (Sun et al., 2026) contrast informative and degraded views to prioritize tokens or trajectories but retain the whole evidence-present teacher as the underlying target. Moreover, positive visual advantage can miss refutation when clear evidence should lower the probability of the student's mistaken token.

These limitations expose a visual-correction attribution problem: estimating which component of the teacher's proposed correction is supported by a controlled visual intervention.

Methodology

On-Policy Counterfactual Views

Let $x^0$ denote the full image available to the student. The student policy $\pi_\theta$ samples a response $y \sim \pi_\theta(\cdot | x^0)$, and all teacher queries reuse the student prefix $y_{<t}$. A fixed copy $\bar{\pi}_\theta$ of the initial model serves as the teacher with two training-only views:

• Evidence-present crop: $x^+$
• Evidence-removed/degraded crop: $x^-$

At position $t$, the three next-token distributions are:

$$p_S^0 = \pi_\theta(\cdot | x^0, y_{<t}), \quad p_T^+ = \bar{\pi}_\theta(\cdot | x^+, y_{<t}), \quad p_T^- = \bar{\pi}_\theta(\cdot | x^-, y_{<t})$$

Each distribution is mapped to a centered log-probability vector:

$$\phi_t(p) = \log(p[V_t] + \epsilon) - \text{mean}(\log(p[V_t] + \epsilon))$$

Correction Attribution

The privileged-teacher correction and its response to the visual intervention are:

$$r_t = \phi_t(p_T^+) - \phi_t(p_S^0), \quad u_t = \phi_t(p_T^+) - \phi_t(p_T^-)$$

The proxy vector $u_t$ captures the visual evidence direction: $u_t(i) > 0$ means revealing evidence raises relative probability, while $u_t(i) < 0$ means evidence refutes it.

The one-sided projection is:

$$\beta_t = \frac{[\langle r_t, u_t \rangle]_+}{\|u_t\|_2^2 + \zeta}, \quad r_t^{\text{vis}} = \beta_t u_t, \quad r_t^{\text{res}} = r_t - r_t^{\text{vis}}$$

Budgeted Support and Refutation

The method decouples support and refutation:

$$u_t^+ = [u_t]_+, \quad u_t^- = [u_t]_-, \quad s_t^\pm = [\langle r_t, u_t^\pm \rangle]_+$$ $$\omega_t^+ = \min\left(\frac{s_t^+}{Z_t}, \tau^+\right), \quad \omega_t^- = \frac{s_t^-}{Z_t}$$

The resulting correction and target are:

$$r_t^{\text{VAD}} = B_t \left( \omega_t^+ \frac{u_t^+}{\|u_t^+\|_2 + \epsilon} + \omega_t^- \frac{u_t^-}{\|u_t^-\|_2 + \epsilon} \right)$$ $$q_{T,t}^{\text{VAD}} = \text{softmax}\left( \phi_t(p_S^0) + \text{clip}(r_t^{\text{VAD}}, -c, c) \right)$$

Training Objective

The primary supervision uses Jensen-Shannon divergence:

$$\mathcal{L}_{\text{vis}} = \frac{1}{|\mathcal{T}|} \sum_{t \in \mathcal{T}} D_{\text{JS}}\left( \text{stopgrad}(q_{T,t}^{\text{VAD}}), p_S^0 \right)$$

A weak regularizer is added to prevent language drift:

$$\rho_t = \frac{\|r_t^{\text{VAD}}\|_2}{\|r_t\|_2 + \epsilon}, \quad a_t = \text{stopgrad}(\text{clip}(1 - \rho_t, 0, 1))$$ $$\mathcal{L} = \mathcal{L}_{\text{vis}} + \lambda \mathcal{L}_{\text{reg}}, \quad \mathcal{L}_{\text{reg}} = \frac{1}{|\mathcal{T}|} \sum_{t \in \mathcal{T}} a_t D_{\text{JS}}\left( \text{stopgrad}(p_T^+), p_S^0 \right)$$

Empirical Validation / Results

Experimental Setup

• Models: Qwen3.5-4B and Qwen3.5-9B
• Training data: 6,241 synthetic visual QA examples from Vision-OPD
• Benchmarks: VStar, ZoomBench, HRBench (4K/8K), MME-RealWorld (EN/CN)
• Baselines: GRPO, VA-OPD, V-Zero, Vision-OPD, Decomposed OPD

RQ1: Fine-Grained Visual Accuracy

Table 1: Overall comparison on six fine-grained visual benchmarks

VAD achieves 78.32 and 79.93 Avg 6 at 4B and 9B respectively, leading the best scale-matched alternative by 2.40 points at 4B and 2.80 points at 9B.

RQ2: Semantics of the Attributed Correction

The attributed direction $r_t^{\text{vis}}$ selectively concentrates visual evidence and decision semantics:

• Visual attributes, objects/content, and A–D decisions account for 42.0% of the relative top-5 composition in $r_t^{\text{vis}}$, compared with 26.7% in $r_t$ and only 17.7% in $r_t^{\text{res}}$
• Language/format and artifact/meta semantics decrease to 58.0% in $r_t^{\text{vis}}$, versus 73.3% in $r_t$ and 82.3% in $r_t^{\text{res}}$

Token-level analysis shows A–D answer symbols, attribute tokens (e.g., vertical, metal) are more prominent in $r_t^{\text{vis}}$, while language scaffolding tokens (e.g., to, inside) are more prominent in $r_t^{\text{res}}$.

RQ3: Effects of Alternative Supervision Targets

Table 2: Target-construction and regularization ablation (Qwen3.5-4B)

Branch-aware target reconstruction provides the main gain (1.00 point from one-sided to VAD), with a smaller benefit from regularization.

RQ4: Generalization of Post-Trained Checkpoints

Table 3: Held-out generalization

VAD is the only post-trained method with positive ∆ at both scales, showing attribution-based reconstruction preserves held-out capability while improving fine-grained perception.

RQ5: Effects of Training Choices

Table 4: Effect of supervision divergence

JSD yields the highest aggregate accuracy at both scales, making it the most balanced default across tasks.

Theoretical and Practical Implications

Theoretical implications:

• The paper identifies "source-mixed teacher correction" as a central limitation of privileged multimodal OPD, showing that direct teacher matching and positive visual-advantage weighting do not explicitly estimate which correction component is supported by a controlled visual intervention.
• VAD introduces a principled framework for counterfactual target reconstruction that separates visually attributable corrections from residual components, providing a more interpretable and targeted supervision signal.
• The signed proxy $u_t$ captures both support and refutation directions, addressing the limitation of positive visual advantage that misses evidence-based suppression of wrong tokens.

Practical implications:

• VAD achieves consistent improvements across six benchmarks and two model scales with only modest additional training cost (3.8% more step time than Vision-OPD at 4B).
• The method requires no additional teacher views or model calls at inference time, making it a practical drop-in replacement for existing OPD approaches.
• The gains are broad rather than benchmark-specific, spanning fine-detail localization, high-resolution perception, and real-world recognition.
• VAD preserves or improves held-out generalization, unlike other post-training methods that often sacrifice broader capability.

Conclusion

VAD addresses source-mixed teacher corrections in multimodal on-policy distillation by contrasting evidence-present and evidence-removed teacher views, attributing the visually aligned correction, and reconstructing a student-anchored target with separate support and refutation budgets. Across six benchmarks, it leads scale-matched methods at 4B and 9B while preserving Base-level held-out performance. Semantic and offline analyses show that the attributed direction concentrates visual and decision-relevant content while strengthening correct-token support and wrong-token suppression.

Limitations:

1. Each intervention is represented by one contrastive vector from a single view pair, which may bias compositional evidence; multiple views or learned directional bases could provide a richer estimate.
2. The current projection yields semantic enrichment rather than an identifiable separation: the attributed component can retain nonvisual teacher effects, and the residual remains source-mixed. Learned decompositions with grounding constraints may yield cleaner attribution.