Summary (Overview)

• Proposes Visual Contrastive Self-Distillation (VCSD), a novel on-policy self-distillation (OPSD) method for vision-language models (VLMs) that constructs teacher–student target asymmetry using only input conditioning, requiring no external teacher, privileged answers, reasoning traces, or visual evidence signals.
• Key mechanism: At each student-generated response prefix, an EMA teacher produces two next-token distributions—one conditioned on the original image and one on a content-erased control (e.g., a black image). Their token-wise log-probability difference $\Delta_t(v)$ is used to sharpen the original-image teacher distribution within a plausible support set.
• The contrast-shaped target $q^*_t(v)$ combines the original-image distribution as an anchor with the contrast signal, formalized as a one-step KL-regularized policy update maximizing an implicit visual-evidence reward.
• Consistent improvements across Qwen3-VL (2B, 4B, 8B) and Qwen3.5 (2B, 4B, 9B) models on a seven-benchmark aggregate (BLINK, MMStar, V* Bench, MathVista, HRBench4K/8K, HallusionBench), outperforming both base models and matched answer-hint OPSD baselines.
• Practical advantages: No additional inference-time cost, no external verifier, and robust to the exact choice of content-erased control image (black, Gaussian noise, blur, or no image).

Introduction and Theoretical Foundation

Background and Motivation: On-policy distillation (OPD) trains a student on prefixes from its own policy using an external teacher, aligning training with inference trajectories but requiring a stronger teacher. On-policy self-distillation (OPSD) removes the external teacher by using an EMA teacher from the same model, but effective OPSD requires a teacher–student asymmetry to make the self-teacher more informative than the student. Existing approaches create this asymmetry through privileged answers, reasoning traces, or visual evidence signals (e.g., image crops), which depend on task-specific information or additional processing pipelines.

Research Question: Can the asymmetry required by on-policy self-distillation be constructed purely from input conditioning, without auxiliary information?

Core Idea: Instead of treating a content-erased image as an alternative to imitate, use it as a controlled reference. At each student-generated prefix, evaluate the EMA teacher twice under the same prompt and prefix—once with the original image $J$ and once with a content-erased control $J_{\text{ctrl}}$ (e.g., a black image). The token-wise log-probability difference $\Delta_t(v)$ captures how each candidate’s likelihood changes when the instance-specific visual content is removed, providing the target asymmetry needed for self-distillation.

Theoretical Insight: The contrast $\Delta_t(v)$ can be interpreted as:

1. A dense token-level reward measuring predictive support attributable to instance-specific visual content.
2. A controlled approximation to conditional pointwise mutual information: $$\text{PMI}(v; J | H_t) = \log \frac{p(v | J, H_t)}{p(v | H_t)}$$ where $H_t = (P, y_{<t})$ is the textual context, and the content-erased prediction $p_\phi(v | J_{\text{ctrl}}, H_t)$ serves as a surrogate for $p(v | H_t)$.

Methodology

On-Policy Self-Distillation Setup: Given prompt–image pairs $(P, J)$, the student $\pi_\theta$ samples an on-policy response $y \sim \pi_\theta(\cdot | P, J)$. The EMA teacher $\pi_\phi$ is evaluated at each student-generated prefix $y_{<t}$ under both visual conditions. The student next-token distribution is $p_{\theta,t}(v) = \pi_\theta(v | P, J, y_{<t})$.

Visual Conditioning Contrast: For a content-erased control $J_{\text{ctrl}} = C(J)$ (e.g., a same-size black RGB image), the teacher produces:

$$p^J_{\phi,t}(v) = \pi_\phi(v | P, J, y_{<t}), \quad p^0_{\phi,t}(v) = \pi_\phi(v | P, J_{\text{ctrl}}, y_{<t})$$

The token-wise log-probability contrast is:

$$\Delta_t(v) = \log p^J_{\phi,t}(v) - \log p^0_{\phi,t}(v) \quad (6)$$

Contrast-Shaped Teacher Target: To avoid amplifying unlikely tokens with large relative changes, the target is restricted to a plausibility support:

$$S_t(\beta) = \left\{ v \in \mathcal{V} : p^J_{\phi,t}(v) \geq \beta \max_{u \in \mathcal{V}} p^J_{\phi,t}(u) \right\} \quad (7)$$

The contrast-shaped target $q^*_t$ is defined as:

$$q^*_t(v) = \frac{\mathbf{1}[v \in S_t(\beta)] \, p^J_{\phi,t}(v) \exp(\alpha \tilde{\Delta}_t(v))}{\sum_{u \in S_t(\beta)} p^J_{\phi,t}(u) \exp(\alpha \tilde{\Delta}_t(u))} \quad (8)$$

where $\alpha \geq 0$ controls contrastive strength and $\tilde{\Delta}_t$ sets contrast of termination tokens to zero. The unnormalized log-score for non-termination tokens is:

$$\log p^J_{\phi,t}(v) + \alpha \tilde{\Delta}_t(v) = (1+\alpha) \log p^J_{\phi,t}(v) - \alpha \log p^0_{\phi,t}(v) \quad (9)$$

On-Policy Distillation and EMA Update: The full-distribution forward KL divergence is minimized:

$$\mathcal{L}_{\text{VCSD}} = T_{\text{KD}}^2 \; \mathbb{E}_{(P,J) \sim \mathcal{D}, \; y \sim \pi_\theta(\cdot | P, J)} \left[ \frac{1}{|y|} \sum_{t=1}^{|y|} D_{\text{KL}}(\text{sg}[q^*_t] \, \| \, p_{\theta,t}) \right] \quad (10)$$

where $\text{sg}[\cdot]$ denotes stop-gradient. The EMA teacher is updated as $\phi \leftarrow \mu\phi + (1-\mu)\theta$ with decay coefficient $\mu$.

Theoretical Perspective (Remark 1): For each prefix, the target $q^*_t$ is the unique solution to:

$$q^*_t = \arg\max_{q \in \Pi(S_t(\beta))} \left\{ \alpha \mathbb{E}_{v \sim q}[ r^{\text{vis}}_t(v) ] - D_{\text{KL}}(q \| \bar{p}^J_{\phi,t}) \right\} \quad (13)$$

where $\bar{p}^J_{\phi,t}$ is the support-normalized original-image distribution and $r^{\text{vis}}_t(v) = \tilde{\Delta}_t(v)$. This shows that VCSD performs a one-step KL-regularized policy update using the contrast as an implicit visual-evidence reward. The derivation is provided in Appendix A.

Empirical Validation / Results

Setup: Evaluated on ViRL39K dataset across Qwen3-VL (2B, 4B, 8B) and Qwen3.5 (2B, 4B, 9B) models. Baselines: base model, answer-hint OPSD (Zhao et al., 2026). Seven benchmarks: BLINK, MMStar, V* Bench, MathVista, HRBench4K/8K, HallusionBench. Aggregate metric: unweighted mean accuracy.

Main Results (Table 1):

• VCSD improves aggregate accuracy by +1.86% to +4.77% over base models and consistently outperforms OPSD.
• On Qwen3.5, OPSD provides no consistent gain, while VCSD improves every tested scale.

Ablation Studies (on Qwen3-VL-2B):

• Plausibility support (Figure 3a): Restricting to $S_t(\beta)$ (default $\beta=0.1$) stabilizes long-horizon training; removing it ($\beta=0$) leads to progressive degradation.
• Contrastive strength (Figure 3b): Performance peaks at $\alpha=1$ and is robust for $\alpha \in [1, 1.5]$; $\alpha=0$ reduces accuracy by 2.33%, $\alpha=2$ drops to no-contrast level.
• Distillation divergence (Table 2): Forward KL achieves 67.04%, outperforming JSD (66.25%) and reverse KL (64.77%).
• Control-image construction (Table 3): Black, Gaussian noise, Gaussian blur, and no image all achieve similar aggregate accuracy (~66.2–67.1%), showing robustness.
• Original-image anchor (Table 4, Figure 3c): Removing the anchor ($\lambda=0$) yields similar accuracy (66.78% vs. 67.04%) but increases language drift (non-target-language tokens) throughout training.

Training Dynamics (Figure 4): VCSD maintains a consistent advantage over OPSD at every training step, with less late-stage degradation on aggregate and per-benchmark metrics (MMStar, MathVista).

Qualitative Analysis (Figures 5, 6): On a MathVista base-ten block question, the base model predicts 6169, OPSD predicts 8519, but VCSD correctly counts seven thousand-cubes and predicts 7519. Token-level contrast scores (Figure 6) show VCSD assigns stronger positive contrast to image-grounded concepts (e.g., roof, shingles, food, beige) compared to base and OPSD.

Theoretical and Practical Implications

Theoretical Implications:

• VCSD demonstrates that target asymmetry in OPSD can be derived purely from matched input conditioning, offering a principled alternative to using privileged answers, visual evidence, or external teachers.
• The contrast-shaped target is formally equivalent to a one-step KL-regularized policy update (Remark 1), where the conditioning log-ratio acts as an implicit visual-evidence reward and the original-image distribution serves as the reference policy.
• The contrast $\Delta_t(v)$ approximates conditional pointwise mutual information, encouraging the student to preserve image-dependent evidence rather than relying on language priors.

Practical Implications:

• VCSD requires no external teacher, privileged answers, reasoning traces, evidence-focused crops, external verifiers, or verifiable rewards—only the original prompt–image pair and the student's on-policy response.
• No additional inference-time cost; only the updated student is retained.
• The method is robust to the choice of content-erased control image (black, noise, blur, or no image), simplifying deployment.
• Consistent improvements across model families (Qwen3-VL and Qwen3.5) and scales (2B to 9B) suggest general applicability.

Conclusion

VCSD introduces a simple yet effective approach to on-policy self-distillation for VLMs by constructing teacher–student asymmetry directly from matched visual conditioning. By contrasting the EMA teacher's predictions under the original image and a content-erased control at each student-generated prefix, VCSD produces a visually informed, full-distribution target without auxiliary information. Experiments across two model families and three scales each demonstrate consistent improvements over base models and existing OPSD baselines on seven vision-language benchmarks. The method's theoretical grounding as a one-step KL-regularized policy update and its practical robustness make it a promising direction for self-improving VLMs.

Future directions may include extending VCSD to video, multi-image, and document understanding settings; exploring other forms of controlled conditioning contrasts; and combining with reinforcement learning or preference optimization for further gains.