Summary (Overview)

• On-policy distillation (OPD) of diffusion models under classifier-free guidance (CFG) is shown to be under-identified at the branch level: matching only the CFG-composed velocity allows positive- and negative-branch errors to cancel each other, creating infinitely many degenerate solutions.
• This ambiguity becomes harmful when the teacher’s negative branch contains privileged information unavailable to the student, leading to a failure mode called Negative Branch Asymmetry (NBA): the positive-branch error decreases while the negative-branch error increases during training.
• The authors propose Positive–Direction Matching (PDM), a branch-aware objective that separately constrains the positive prediction and the CFG conditional direction, eliminating cross-branch error compensation.
• PDM is applied to dense-to-sparse video control (pose, depth, scribble), where naive guided matching exhibits severe sensitivity to inference guidance scale, while PDM and a simpler baseline (Independent Branch Matching, IBM) remain stable and improve control fidelity.
• Key insight: NBA is diagnosed by excess degradation relative to the teacher or a branch-aware student under guidance-scale shifts, not by an absolute CFG-free performance drop.

Introduction and Theoretical Foundation

Classifier-free guidance (CFG) (Ho & Salimans, 2022) is the standard inference mechanism for modern diffusion models. The denoising velocity is a composition of two branch predictions:

$$\tilde{v} = \gamma v^+ + (1 - \gamma) v^-$$

where $v^+$ is the positive branch (under target condition) and $v^-$ is the negative branch (under null or negative condition), with guidance scale $\gamma > 1$.

On-policy distillation (OPD) trains a student model by querying a teacher along the student’s own denoising trajectory. The natural extension of OPD to CFG-based models is to match the composed velocities:

$$\mathcal{L}_{\text{naive}} = \mathbb{E}_{x_t^S \sim p_\theta, t} \left[ w(t) \| \gamma e^+ + (1-\gamma) e^- \|_2^2 \right]$$

where $e^+ = v_T^+ - v_S^+$ and $e^- = v_T^- - v_S^-$ are the positive- and negative-branch errors.

This objective is under-identified at the branch level: the condition $\gamma e^+ + (1-\gamma) e^- = 0$ admits infinitely many non-zero pairs $(e^+, e^-)$, allowing the two errors to compensate without changing the composed prediction. The paper investigates when this ambiguity becomes harmful.

Methodology

Negative Branch Asymmetry (NBA)

Under privileged negative conditioning (e.g., teacher’s negative branch retains reference images or dense control signals unavailable to the student), the teacher’s negative target cannot be directly reproduced by the student. The composed objective then induces antagonistic branch-error dynamics: reducing $\|e^+\|_2$ while increasing $\|e^-\|_2$. This failure mode is named NBA.

PDM: Positive–Direction Matching

To prevent cross-branch error compensation, PDM supervises the predictions before CFG composition. Define the CFG conditional direction:

$$d_M = v_M^+ - v_M^-$$

The PDM loss is:

$$\ell_{\text{PDM}} = \|e^+\|_2^2 + \lambda \| d_T - d_S \|_2^2 = \|e^+\|_2^2 + \lambda \| e^+ - e^- \|_2^2$$

where $\lambda > 0$ controls direction-matching strength. Zero PDM loss implies $e^+ = e^- = 0$, eliminating compensation freedom.

Baseline: Independent Branch Matching (IBM)

IBM directly matches both branches with per-branch weights:

$$\ell_{\text{IBM}} = \gamma^2 \|e^+\|_2^2 + (\gamma-1)^2 \|e^-\|_2^2$$

Like PDM, it removes cross-branch interaction but uses a different parameterization.

Efficient Supervision for Video

For dense-to-sparse video control, the authors supervise only the first $K$ student-visited states, reducing cost while maintaining performance:

$$\mathcal{L}_{\text{OPD}}^{(K)} = \mathbb{E} \left[ \frac{1}{K} \sum_{k=0}^{K-1} w(t_k) \ell(x_{t_k}^S, t_k) \right]$$

Empirical Validation / Results

Branch-Error Dynamics (Figure 2)

Two image-domain distillation settings are contrasted:

• Shared negative conditioning (text-rendering): both branch errors decrease jointly under all objectives (benign regime).
• Privileged negative conditioning (reference-conditioned): naive matching reduces $\|e^+\|_2$ while increasing $\|e^-\|_2$ — NBA. PDM suppresses this negative-branch degradation.

Intrinsic vs. Distillation-Induced CFG Sensitivity

Table 1 (text-rendering) shows that naive and PDM students closely track the teacher’s intrinsic CFG response — no excess sensitivity introduced.

In reference-conditioned distillation (Figure 3), naive matching departs from the teacher at off-scale γ, producing visual distortion and style drift, while PDM remains stable.

Main Results on Dense-to-Sparse Video Control

Table 2 reports results at γ=5 across three modalities. PDM provides the most consistent improvements over naive OPD, SFT, and off-policy distillation.

Similar patterns hold for depth and scribble modalities.

Guidance-Scale Generalization

Table 3 shows that naive OPD degrades sharply when inference γ differs from training γ=5 (e.g., MPJPE All 8.98 at γ=1), while PDM and IBM remain stable.

Ablation Studies

Table 4 (direction-matching weight λ): λ=1 provides best control fidelity; larger λ improves distributional quality but weakens control.

Table 5 (supervision horizon K): K=8 gives strongest control fidelity while being substantially cheaper than full-trajectory (K=50) supervision.

Theoretical and Practical Implications

• Theoretical: The paper identifies a previously unrecognized failure mode in CFG-based OPD — NBA — and shows that branch-level ambiguity is not always harmful but becomes problematic under privileged negative conditioning. This provides a principled understanding of when and why naive guided matching fails.
• Practical: PDM is a simple, drop-in replacement for the naive OPD loss that requires no architectural change. It improves robustness to inference guidance-scale shifts and control fidelity in dense-to-sparse video control. The efficient supervision strategy (supervising only early steps) makes OPD practical for video models.
• Broader impact: The findings apply to any OPD setting where the teacher’s CFG schema retains privileged information in the negative branch (e.g., reference images, dense control signals, auxiliary modalities). Branch-aware objectives should be preferred in such cases.

Conclusion

The paper demonstrates that CFG-composed OPD is under-identified at the branch level, leading to NBA under privileged negative conditioning. PDM resolves this by separately supervising the positive prediction and CFG conditional direction. Experiments on text-rendering, reference-conditioned distillation, and dense-to-sparse video control confirm that PDM improves guidance-scale robustness and knowledge transfer.

Limitations: The relative behavior of PDM and IBM is not theoretically characterized; PDM’s advantage is empirical. Future work may extend NBA analysis to other guidance mechanisms and teacher–student conditioning asymmetries.