# UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

> Multi-teacher on-policy distillation with platform-conditioned routing achieves state-of-the-art cross-platform GUI performance without catastrophic forgetting.

- **Source:** [arXiv](https://arxiv.org/abs/2607.04425)
- **Published:** 2026-07-08
- **Permalink:** https://picx.dev/p/I4zBNo
- **Whiteboard:** https://picx.dev/p/I4zBNo/image

## Summary

## Summary (Overview)

- Proposes **UI-MOPD**, the first method to introduce **multi-teacher on-policy distillation (MOPD)** into continual learning for GUI agents, addressing behavioral pattern mixing and catastrophic forgetting when training across heterogeneous desktop and mobile environments.
- Constructs **Uni-GUI**, a high-quality cross-platform GUI interaction dataset consisting of ~10K executable trajectories collected from desktop and mobile environments using a unified data-collection harness.
- Achieves state-of-the-art balanced performance on **OSWorld (38.2%) and MobileWorld (12.0%)**, outperforming model merging, mixed supervised fine-tuning, and existing multi-platform GUI agents.
- Introduces **platform-conditioned teacher routing** and **adaptive KL masking** to preserve platform-specific interaction behaviors while improving task success through online reinforcement learning.
- Demonstrates that UI-MOPD preserves general GUI grounding ability (e.g., ScreenSpot-Pro: 43.14%, AndroidControl⋆: 80.05%) better than static parameter merging, avoiding degradation of foundational capabilities.

---

## Introduction and Theoretical Foundation

### Background
Graphical user interface (GUI) agents have evolved from single-platform web navigation or mobile automation toward cross-platform interaction — spanning computer applications, mobile apps, and web services. Benchmarks such as OSWorld (desktop) and MobileWorld (mobile) evaluate agents in long-horizon, platform-grounded interactive environments.

### Challenges
Building a single GUI agent that operates across platforms faces two fundamental bottlenecks:

1. **Scarcity of high-quality cross-platform trajectories**: Existing datasets (e.g., OpenCUA, OpenMobile) often focus on single platforms, contain invalid actions, or have inconsistent state-action alignment.
2. **Distinct interaction conventions across platforms**: Desktop actions (mouse clicks, keyboard shortcuts, window switching) differ fundamentally from mobile actions (tap, swipe, long press). Naively combining such heterogeneous data via mixed supervised fine-tuning (SFT) or model merging produces an **averaged policy** that degrades platform-specific capabilities and causes **catastrophic forgetting** during continual learning.

### Theoretical Motivation
The core insight is to formulate multi-teacher knowledge integration as a **conditional behavioral constraint** during online policy optimization. Instead of mixing teacher signals indiscriminately, UI-MOPD aligns rollouts from each platform with a dedicated platform-specific teacher. This provides **stable behavioral anchors** that prevent interference between heterogeneous interaction patterns while still allowing the shared policy to discover higher-reward behaviors via reinforcement learning.

---

## Methodology

### 1. Uni-GUI Dataset Construction
A **unified cross-platform data-collection harness** was built to collect interaction data from desktop and mobile environments with a consistent action interface and logging format. After rigorous filtering:

| Platform | Raw Steps | Final High-Quality Trajectories |
|----------|-----------|--------------------------------|
| Desktop  | ~110K     | ~10K total (combined)          |
| Mobile   | ~50K      | ~10K total (combined)          |

### 2. Two-Stage Training Pipeline

**Stage 1: Supervised Fine-Tuning (SFT)**  
Fine-tune a vision-language model (Qwen3-VL-32B-Thinking) on Uni-GUI trajectories per platform to obtain:

- Desktop teacher $\pi^d_{\text{ref}}$
- Mobile teacher $\pi^m_{\text{ref}}$

**Stage 2: Multi-Teacher On-Policy Distillation (MOPD)**  
Train a **shared student policy** $\pi_\theta$ (initialized from Qwen3-VL-8B-Thinking) using online reinforcement learning combined with platform-conditioned distillation.

### 3. On-Policy Kullback–Leibler (KL) Distillation

For a rollout $y^{(i)}$ from the student, and given the platform-routed teacher $\pi^{(i)}_{\text{ref}}$, the per-token reverse KL divergence is:

$$D^{(t,i)}_{\text{KL}} = D_{\text{KL}}\left(\pi_\theta(\cdot|h^{(i)}_t) \parallel \pi^{(i)}_{\text{ref}}(\cdot|h^{(i)}_t)\right)$$

Equivalently:
$$D^{(t,i)}_{\text{KL}} = \mathbb{E}_{a \sim \pi_\theta(\cdot|h^{(i)}_t)}\left[\log \pi_\theta(a|h^{(i)}_t) - \log \pi^{(i)}_{\text{ref}}(a|h^{(i)}_t)\right]$$

**K3 Estimator** (unbiased, low-variance):
Define $\delta^{(i)}_t = \log \pi^{(i)}_{\text{ref}}(y_t|h^{(i)}_t) - \log \pi_\theta(y_t|h^{(i)}_t)$ and $\rho^{(i)}_t = \exp(\delta^{(i)}_t)$. Then:
$$\hat{D}^{(t,i)}_{\text{KL}} = \rho^{(i)}_t - \delta^{(i)}_t - 1$$

**Adaptive KL Masking**:
$$\mu^{(i)} = \begin{cases} 0, & \text{if } \frac{1}{G}\sum_{k \in g^{(i)}}R^{(k)} > \tau_{\text{KL}} \\ 1, & \text{otherwise} \end{cases}$$
removes teacher penalty on rollouts that already receive sufficient reward.

### 4. Platform-Conditioned Teacher Routing

$$\pi^{(i)}_{\text{ref}} = \begin{cases} \pi^m_{\text{ref}}, & s_i \in S_{\text{mobile}} \\ \pi^d_{\text{ref}}, & s_i \in S_{\text{desktop}} \end{cases}$$

The **student remains a single shared policy** — no multiple agents or extra teachers at inference.

### 5. Reward Design

Structured outcome reward for GUI actions:
$$R(x, y) = \begin{cases} 1.0, & f_a = 1 \\ -0.5, & 0 \leq f_a < 1 \\ -1.0, & \text{unparsable/invalid} \end{cases}$$
where $f_a \in [0,1]$ is the fraction of matched action dimensions.

### 6. Training Objective

Maximize the regularized objective:
$$J(\theta) = \mathbb{E}_{p,x,y \sim \pi_\theta}\left[\sum_t m_t \left( \ell^{(t)}_{\text{PG}}(\theta) - \beta \mu \hat{D}^{(t,p)}_{\text{KL}} \right)\right]$$

where $\ell^{(t)}_{\text{PG}}(\theta) = \min\left(r_t(\theta)A_t,\ \text{clip}(r_t(\theta), 1-\epsilon_{\text{low}}, 1+\epsilon_{\text{high}})A_t\right)$ and $A_t$ is group-relative advantage.

Equivalently, minimize:
$$\mathcal{L}(\theta) = \mathcal{L}_{\text{PG}}(\theta) + \beta \mathcal{L}_{\text{MOPD}}(\theta)$$

---

## Empirical Validation / Results

### Main Results (Table 1)

| Method | OSWorld | MobileWorld |
|--------|---------|-------------|
| General Models | | |
| Qwen3-VL-8B-Thinking | 33.9% | 7.7% |
| Qwen3-VL-32B-Instruct | 32.6% | 9.0% |
| **GUI Models (Multi-Platform)** | | |
| GUI-Owl-7B | 34.9% | 4.5% |
| GELab-Zero-4B | 31.9% | 10.9% |
| **Integration Strategies** | | |
| Mixed-SFT | 35.0% | 6.4% |
| Model Merge (TIES) | 36.8% | 0% |
| **UI-MOPD (Ours)** | **38.2%** | **12.0%** |

- UI-MOPD achieves **12.7% relative improvement** over base model on OSWorld and **55.8%** on MobileWorld.
- Outperforms all integration baselines, especially on MobileWorld where model merging completely fails (0%).

### Teacher-Student Analysis (Table 2)

| Model | OSWorld | MobileWorld |
|-------|---------|-------------|
| Qwen3-VL-8B-Thinking (base) | 33.9% | 7.7% |
| 8B SFT on OSWorld only | 35.8% | 0% |
| 8B SFT on MobileWorld only | 35.8% | 12.8% |
| Desktop Teacher (32B) | 46.3% | – |
| Mobile Teacher (32B) | – | 16.2% |
| **UI-MOPD (8B student)** | **38.2%** | **12.0%** |

- Single-platform SFT leads to severe capability loss on the other platform (0% on MobileWorld after desktop-only SFT).
- UI-MOPD improves **both platforms simultaneously** by +4.3 points each.

### General GUI Grounding (Table 3)

| Model | AndroidControl⋆ | ScreenSpot-Pro | ScreenSpotV2 | OSWorld-G |
|-------|-----------------|----------------|--------------|-----------|
| Qwen3-VL-8B-Thinking | 78.73% | 43.71% | 91.27% | 52.13% |
| Model Merge (TIES) | 74.01% | 37.13% | 88.60% | 47.16% |
| **UI-MOPD** | **80.05%** | **43.14%** | **90.88%** | **52.84%** |

- UI-MOPD **preserves** or slightly improves grounding ability, whereas model merging causes consistent degradation.

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Platform-conditioned distillation as a behavioral anchor**: The paper provides formal evidence that routing teacher supervision by platform prevents the "averaged policy" problem inherent in mixed training or static merging. The KL divergence acts not as a conservative regularizer (as in standard RLHF) but as a **targeted mechanism for transferring platform-specific expertise** while allowing RL to improve task success.

2. **On-policy nature is crucial**: Distilling from the student's own rollouts (not static offline trajectories) concentrates supervision on states the student actually struggles with, making the transfer more efficient and less destructive.

3. **Adaptive KL masking**: The discovery that teacher supervision is most beneficial when task rewards are still weak suggests a principled schedule for balancing exploration vs. behavioral preservation.

### Practical Implications

1. **Unified cross-platform agent**: UI-MOPD demonstrates that a single small (8B) model can match or exceed much larger models (32B) on both desktop and mobile benchmarks, reducing inference cost and deployment complexity.

2. **Mitigating catastrophic forgetting**: The method provides a practical recipe for continually adding new platforms without sacrificing performance on previously learned ones — a critical requirement for real-world agent deployment.

3. **Data efficiency**: The Uni-GUI dataset (~10K trajectories) shows that quality and platform-aware training methodology matter more than raw data scale.

4. **Generalization to other domains**: The MOPD framework is general and could be applied to any multi-environment agent (robotics, web, API) where different environments have distinct behavioral conventions.

---

## Conclusion

This work tackles the problem of continual learning for multi-platform GUI agents. The key contributions are:

- **Uni-GUI dataset**: High-quality cross-platform interaction trajectories from desktop and mobile environments.
- **UI-MOPD method**: First introduction of multi-teacher on-policy distillation (MOPD) for GUI agents, featuring platform-conditioned teacher routing, K3 KL estimation, and adaptive KL masking.

Experimental results on OSWorld (38.2%) and MobileWorld (12.0%) show substantial improvements over model merging and mixed SFT baselines, while preserving general GUI grounding ability.

**Future directions** include:
- Extending UI-MOPD to more platforms (e.g., web, automotive, IoT).
- Integrating human feedback for reward design.
- Scaling to larger teacher ensembles.
- Exploring dynamic teacher routing without explicit platform labels.

---

_Markdown view of https://picx.dev/p/I4zBNo, served by PicX — AI-generated visual whiteboard summaries of research papers._