Summary (Overview)

• Proposes UI-MOPD, the first method to introduce multi-teacher on-policy distillation (MOPD) into continual learning for GUI agents, addressing behavioral pattern mixing and catastrophic forgetting when training across heterogeneous desktop and mobile environments.
• Constructs Uni-GUI, a high-quality cross-platform GUI interaction dataset consisting of ~10K executable trajectories collected from desktop and mobile environments using a unified data-collection harness.
• Achieves state-of-the-art balanced performance on OSWorld (38.2%) and MobileWorld (12.0%), outperforming model merging, mixed supervised fine-tuning, and existing multi-platform GUI agents.
• Introduces platform-conditioned teacher routing and adaptive KL masking to preserve platform-specific interaction behaviors while improving task success through online reinforcement learning.
• Demonstrates that UI-MOPD preserves general GUI grounding ability (e.g., ScreenSpot-Pro: 43.14%, AndroidControl⋆: 80.05%) better than static parameter merging, avoiding degradation of foundational capabilities.

---

Introduction and Theoretical Foundation

Background

Graphical user interface (GUI) agents have evolved from single-platform web navigation or mobile automation toward cross-platform interaction — spanning computer applications, mobile apps, and web services. Benchmarks such as OSWorld (desktop) and MobileWorld (mobile) evaluate agents in long-horizon, platform-grounded interactive environments.

Challenges

Building a single GUI agent that operates across platforms faces two fundamental bottlenecks:

1. Scarcity of high-quality cross-platform trajectories: Existing datasets (e.g., OpenCUA, OpenMobile) often focus on single platforms, contain invalid actions, or have inconsistent state-action alignment.
2. Distinct interaction conventions across platforms: Desktop actions (mouse clicks, keyboard shortcuts, window switching) differ fundamentally from mobile actions (tap, swipe, long press). Naively combining such heterogeneous data via mixed supervised fine-tuning (SFT) or model merging produces an averaged policy that degrades platform-specific capabilities and causes catastrophic forgetting during continual learning.

Theoretical Motivation

The core insight is to formulate multi-teacher knowledge integration as a conditional behavioral constraint during online policy optimization. Instead of mixing teacher signals indiscriminately, UI-MOPD aligns rollouts from each platform with a dedicated platform-specific teacher. This provides stable behavioral anchors that prevent interference between heterogeneous interaction patterns while still allowing the shared policy to discover higher-reward behaviors via reinforcement learning.

---

Methodology

1. Uni-GUI Dataset Construction

A unified cross-platform data-collection harness was built to collect interaction data from desktop and mobile environments with a consistent action interface and logging format. After rigorous filtering:

2. Two-Stage Training Pipeline

Stage 1: Supervised Fine-Tuning (SFT)
Fine-tune a vision-language model (Qwen3-VL-32B-Thinking) on Uni-GUI trajectories per platform to obtain:

• Desktop teacher $\pi^d_{\text{ref}}$
• Mobile teacher $\pi^m_{\text{ref}}$

Stage 2: Multi-Teacher On-Policy Distillation (MOPD)
Train a shared student policy $\pi_\theta$ (initialized from Qwen3-VL-8B-Thinking) using online reinforcement learning combined with platform-conditioned distillation.

3. On-Policy Kullback–Leibler (KL) Distillation

For a rollout $y^{(i)}$ from the student, and given the platform-routed teacher $\pi^{(i)}_{\text{ref}}$, the per-token reverse KL divergence is:

$$D^{(t,i)}_{\text{KL}} = D_{\text{KL}}\left(\pi_\theta(\cdot|h^{(i)}_t) \parallel \pi^{(i)}_{\text{ref}}(\cdot|h^{(i)}_t)\right)$$

Equivalently:

$$D^{(t,i)}_{\text{KL}} = \mathbb{E}_{a \sim \pi_\theta(\cdot|h^{(i)}_t)}\left[\log \pi_\theta(a|h^{(i)}_t) - \log \pi^{(i)}_{\text{ref}}(a|h^{(i)}_t)\right]$$

K3 Estimator (unbiased, low-variance): Define $\delta^{(i)}_t = \log \pi^{(i)}_{\text{ref}}(y_t|h^{(i)}_t) - \log \pi_\theta(y_t|h^{(i)}_t)$ and $\rho^{(i)}_t = \exp(\delta^{(i)}_t)$. Then:

$$\hat{D}^{(t,i)}_{\text{KL}} = \rho^{(i)}_t - \delta^{(i)}_t - 1$$

Adaptive KL Masking:

$$\mu^{(i)} = \begin{cases} 0, & \text{if } \frac{1}{G}\sum_{k \in g^{(i)}}R^{(k)} > \tau_{\text{KL}} \\ 1, & \text{otherwise} \end{cases}$$

removes teacher penalty on rollouts that already receive sufficient reward.

4. Platform-Conditioned Teacher Routing

$$\pi^{(i)}_{\text{ref}} = \begin{cases} \pi^m_{\text{ref}}, & s_i \in S_{\text{mobile}} \\ \pi^d_{\text{ref}}, & s_i \in S_{\text{desktop}} \end{cases}$$

The student remains a single shared policy — no multiple agents or extra teachers at inference.

5. Reward Design

Structured outcome reward for GUI actions:

$$R(x, y) = \begin{cases} 1.0, & f_a = 1 \\ -0.5, & 0 \leq f_a < 1 \\ -1.0, & \text{unparsable/invalid} \end{cases}$$

where $f_a \in [0,1]$ is the fraction of matched action dimensions.

6. Training Objective

Maximize the regularized objective:

$$J(\theta) = \mathbb{E}_{p,x,y \sim \pi_\theta}\left[\sum_t m_t \left( \ell^{(t)}_{\text{PG}}(\theta) - \beta \mu \hat{D}^{(t,p)}_{\text{KL}} \right)\right]$$

where $\ell^{(t)}_{\text{PG}}(\theta) = \min\left(r_t(\theta)A_t,\ \text{clip}(r_t(\theta), 1-\epsilon_{\text{low}}, 1+\epsilon_{\text{high}})A_t\right)$ and $A_t$ is group-relative advantage.

Equivalently, minimize:

$$\mathcal{L}(\theta) = \mathcal{L}_{\text{PG}}(\theta) + \beta \mathcal{L}_{\text{MOPD}}(\theta)$$

---

Empirical Validation / Results

Main Results (Table 1)

• UI-MOPD achieves 12.7% relative improvement over base model on OSWorld and 55.8% on MobileWorld.
• Outperforms all integration baselines, especially on MobileWorld where model merging completely fails (0%).

Teacher-Student Analysis (Table 2)

• Single-platform SFT leads to severe capability loss on the other platform (0% on MobileWorld after desktop-only SFT).
• UI-MOPD improves both platforms simultaneously by +4.3 points each.

General GUI Grounding (Table 3)

• UI-MOPD preserves or slightly improves grounding ability, whereas model merging causes consistent degradation.

---

Theoretical and Practical Implications

Theoretical Implications

1. Platform-conditioned distillation as a behavioral anchor: The paper provides formal evidence that routing teacher supervision by platform prevents the "averaged policy" problem inherent in mixed training or static merging. The KL divergence acts not as a conservative regularizer (as in standard RLHF) but as a targeted mechanism for transferring platform-specific expertise while allowing RL to improve task success.

2. On-policy nature is crucial: Distilling from the student's own rollouts (not static offline trajectories) concentrates supervision on states the student actually struggles with, making the transfer more efficient and less destructive.

3. Adaptive KL masking: The discovery that teacher supervision is most beneficial when task rewards are still weak suggests a principled schedule for balancing exploration vs. behavioral preservation.

Practical Implications

1. Unified cross-platform agent: UI-MOPD demonstrates that a single small (8B) model can match or exceed much larger models (32B) on both desktop and mobile benchmarks, reducing inference cost and deployment complexity.

2. Mitigating catastrophic forgetting: The method provides a practical recipe for continually adding new platforms without sacrificing performance on previously learned ones — a critical requirement for real-world agent deployment.

3. Data efficiency: The Uni-GUI dataset (~10K trajectories) shows that quality and platform-aware training methodology matter more than raw data scale.

4. Generalization to other domains: The MOPD framework is general and could be applied to any multi-environment agent (robotics, web, API) where different environments have distinct behavioral conventions.

---

Conclusion

This work tackles the problem of continual learning for multi-platform GUI agents. The key contributions are:

• Uni-GUI dataset: High-quality cross-platform interaction trajectories from desktop and mobile environments.
• UI-MOPD method: First introduction of multi-teacher on-policy distillation (MOPD) for GUI agents, featuring platform-conditioned teacher routing, K3 KL estimation, and adaptive KL masking.

Experimental results on OSWorld (38.2%) and MobileWorld (12.0%) show substantial improvements over model merging and mixed SFT baselines, while preserving general GUI grounding ability.

Future directions include:

• Extending UI-MOPD to more platforms (e.g., web, automotive, IoT).
• Integrating human feedback for reward design.
• Scaling to larger teacher ensembles.
• Exploring dynamic teacher routing without explicit platform labels.