# Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents

> GUI agent trajectories should be split into four components and routed to weights or context based on recurrence and state-conditionality, outperforming whole-trajectory approaches.

- **Source:** [arXiv](https://arxiv.org/abs/2610.01787)
- **Published:** 2026-10-17
- **Permalink:** https://picx.dev/p/gAnBHT
- **Whiteboard:** https://picx.dev/p/gAnBHT/image

## Summary

## Summary (Overview)

- **Component routing framework**: The paper introduces a method that decomposes GUI agent trajectories into four distinct components—locators (G), procedures (P), state facts (F), and lessons (L)—and routes each to either the model's context (prompt) or weights (fine-tuning) based on two measurable properties: **recurrence** ($r$) and **state-conditionality** ($\kappa$).

- **Key empirical finding**: Locators (+4.9 points) and lessons (+1.5) perform better in the weights, while procedures (−4.2) and state facts (−4.1) perform better in the context—a consistent pattern across three backbone families (GUI-Owl-8B, Qwen3-VL-8B, ScaleCUA-7B) and two environments (MobileGym, AndroidWorld).

- **Predictive rule**: A linear decision rule in $(r, \kappa)$ recovers the correct destination for 24 of 24 held-out backbone cells, and routing by this rule beats every whole-trajectory baseline, exceeding the better single destination by +3.5 points on average.

- **Mechanistic insights**: Training on a component makes its notes redundant in context (readout drops from ~42% to ~16% for high-recurrence components); context gains scale with the **information gap** between producer and consumer, while weights gains are eroded by the **policy gap**.

- **Producer–consumer analysis**: Notes (context route) transfer effectively across producers regardless of identity, while fine-tuning (weights route) only helps when producer and consumer are the same model, with most of its gain attributable to practice on the consumer's own trajectories.

---

## Introduction and Theoretical Foundation

### Background and Motivation

Self-improving GUI agents (agents that operate phones and desktops) generate trajectories during task execution, which can be reused to improve future performance through two primary mechanisms:

1. **Fine-tuning (weights route)**: Distilling successful trajectories into model parameters
2. **Retrieval (context route)**: Storing trajectories in an external library retrieved into the prompt

Previous comparisons of these two destinations have reached **contradictory conclusions**—some studies find internalization (weights) superior, others find prompt-side updates superior. The authors attribute this disagreement to the **unit of experience**: treating whole trajectories as indivisible units bundles items with fundamentally different properties, making conclusions dependent on the composition of the trajectory mix.

### Core Research Question

> Which components of an agent's own experience belong in its context and which in its weights, and does the answer survive training and a change of backbone?

### Theoretical Framework

The paper defines two pool-level properties measurable **before any training**:

**Recurrence** ($r$): How often a component's key appears in unrelated tasks—measuring cross-task transferability:
$$r(d) \triangleq \frac{1}{|P_d|} \sum_{p \in P_d} \frac{1}{|N_p|} \sum_{q \in N_p} \mathbb{1}[k(d) \in K_q]$$

**State-conditionality** ($\kappa$): How much a component's usefulness depends on the current state matching its source state. The proxy uses state signatures with Jaccard similarity tiers (exact: $J(x,y) \geq 0.9$; relaxed: $0.5 \leq J(x,y) < 0.9$):
$$\kappa_0(d) \triangleq 1 - \frac{a_R(d)}{a_E(d)}$$

where $a_\tau(d)$ measures payload agreement (position within 0.5% of screen diagonal, or target identity) across state-matching occurrence pairs at tier $\tau$.

---

## Methodology

### Problem Formulation

An agent with policy $\pi$ runs tasks and keeps all trajectories (successful or not), forming experience pool $E$. An extractor maps $E$ to four item sets $D_c$ for $c \in \{G, P, F, L\}$. The **route gain** measures the difference between destinations:

$$\Delta(c) \triangleq U(W_c) - U(C_c) = \frac{1}{|T|} \sum_{x \in T} \left[ u(x; W_c) - u(x; C_c) \right]$$

where $\Delta(c) > 0$ indicates the weights destination is superior.

### Component Definitions

| Component | Key | Note (context) | Sample (weights) |
|-----------|-----|----------------|-------------------|
| **Locator (G)** | App, role, element name | Element position | Screenshot → click pair |
| **Procedure (P)** | Three consecutive targets | Subgoal sequence | Next-action prediction with subgoal thought |
| **State fact (F)** | App, target, predicate | Precondition for action | Action prediction with predicate thought |
| **Lesson (L)** | App, state, failed target | Corrective guidance | Preference pair (failed vs. succeeded) |

### Two Destinations

- **Context route ($C_c$)**: Items indexed by application and key; at each step, top-5 items ranked by state-similarity are appended to the prompt as notes (exact matches first)
- **Weights route ($W_c$)**: Fine-tuning via LoRA—supervised fine-tuning for G, P, F; direct preference optimization for L; all trained arms reach ≥0.90 token accuracy on held-in development split

### Routing Rule

A linear score in the two properties, fitted with one backbone family held out:
$$s_\theta(c) \triangleq \alpha r(c) - \beta \kappa_0(c) - \gamma$$

The held-out rule recovers the sign of $\Delta(c)$ with $\alpha \in [9.01, 9.32]$, $\beta \in [4.86, 5.28]$, $\gamma \in [0.70, 0.94]$. A cell routes to weights if $s_f(c) > 0$, else to context.

### Evaluation Setup

- **MobileGym**: Simulated phone, 20 apps, 416 task templates (160 train/256 test); 122 training templates form the pool; 1,106 paired held-out instances per seed
- **AndroidWorld**: 116 tasks on 20 real apps; 12 apps provide experience, 8 held out; 232 paired instances per seed
- Three seeds control pool sampling, training, and instance sampling; paired bootstrap over instances (10,000 resamples, 95% intervals)

---

## Empirical Validation / Results

### Observation 1: Components Split by Destination

The route gain pattern is consistent across backbones and environments:

| Component | Weights gain | Context gain | Route gain $\Delta$ |
|-----------|-------------|--------------|---------------------|
| Locators (G) | +6.7 | +1.8 | **+4.9** (weights) |
| Procedures (P) | +1.0 | +5.2 | **−4.2** (context) |
| State facts (F) | ~+0.8 | ~+4.9 | **−4.1** (context) |
| Lessons (L) | ~+2.5 | ~+1.0 | **+1.5** (weights, weak) |

### Observation 2: Producer–Consumer Asymmetry

- **Context route**: Notes help every consumer regardless of producer; larger producers help more (8B consumer gains +3.6 from own procedures, +4.8 from 32B's, +2.2 from 4B's)
- **Weights route**: Fine-tuning only helps with the same model as producer; other-family procedures change the 8B consumer by **−7.5 points**
- Most weights gain on own procedures is **practice** (+0.6 from own successes alone vs. +0.9 with all samples)

### Observation 3: Readout After Training

The readout probe replays decision steps with/without retrieved notes:

- Before training: notes change actions in 42% of probed steps
- After training on same items: rate falls to 16% (locators), 15% (lessons), but only 22% (procedures)
- After-to-before ratio falls with recurrence (Spearman **−0.92**); splitting items by recurrence thirds shows ratio falling from 0.73 to 0.11 (locators) and 0.63 to 0.11 (lessons)

### Interventions

| Intervention | Component | Effect on route gain |
|-------------|-----------|---------------------|
| Items copied ×8 (increase dose/recurrence) | Procedures | −4.2 → **−0.8** |
| Items copied ×8 | State facts | +3.6 (reaches context on Qwen3-VL) |
| State condition removed (task-text retrieval) | Procedures | Context gain drops **3.9 points** |
| State condition removed | State facts | Context gain drops **3.4 points** |
| Random notes from other apps | — | −1.6 points (control) |

### Routing Against Alternatives

| Arm | MobileGym (avg) | AndroidWorld (avg) | Δ vs. routing |
|-----|-----------------|---------------------|---------------|
| Base agent | 19.1 | 21.8 | ↑ 14.1 |
| Self-retry (3 attempts) | 23.9 | 25.3 | ↑ 9.3 |
| Whole-trajectory SFT | 19.8 | 22.2 | ↑ 13.4 |
| Whole-trajectory retrieval | 18.0 | 21.0 | ↑ 15.2 |
| All → weights | 28.0 | 28.9 | ↑ 5.2 |
| All → context | 28.5 | 30.2 | ↑ 4.7 |
| **Routing (ours)** | **33.2** | **33.0** | **—** |
| Reverse assignment | 24.3 | 26.3 | ↑ 8.9 |

Routing exceeds the better single destination by +4.4 points on MobileGym and +3.2 to +5.0 on AndroidWorld per backbone.

### The Three Gaps

$$\text{Information gap: } g_I(p,q,\phi) \triangleq \sigma_\phi(p)[1 - \sigma_\phi(q)]$$
$$\text{Policy gap: } g_P(p,q) \triangleq \frac{1}{|X(p,q)|} \sum_{x \in X(p,q)} \mathbb{1}[a_q(x) \neq a_p(x)]$$
$$\text{Producer ability: } g_A(p) \triangleq U_0(p)$$

- Context gain rises with information gap: +0.2 (lowest sixth) → **+11.3** (highest sixth), Spearman 0.65
- Weights gain falls with policy gap: +4.0 (lowest sixth) → **−5.7** (highest sixth), Spearman −0.52
- Information and policy gaps vary almost independently (Spearman −0.13)

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Unit of experience matters**: The disagreement between prior studies comparing fine-tuning vs. retrieval stems from treating trajectories as indivisible bundles. Component-level analysis resolves the contradiction—different components have genuinely different optimal destinations.

2. **Predictable routing**: Two simple, pre-training-measurable properties (recurrence, state-conditionality) predict the optimal destination, suggesting a principled theory of experience reuse in agents: **recurring, state-independent knowledge belongs in parameters; rare, state-dependent knowledge belongs in context**.

3. **Mechanistic account of redundancy**: The readout drop after training quantifies how weights absorb recurring knowledge, making context notes redundant—a measurable complementarity between the two memory systems.

4. **Producer–consumer theory**: The information/policy gap decomposition explains when transfer works: context route pays for information the consumer lacks; weights route pays only for policy agreement (same model) and mostly for practice.

### Practical Implications

- **Architecture guidance**: GUI agents should maintain dual memory systems—parameter-embedded knowledge for locators and lessons, external note libraries for procedures and state facts
- **Transfer protocols**: Note libraries survive consumer upgrades and cross-producer transfer; weights must be rebuilt from the consumer's own trajectories
- **Cost efficiency**: Routing avoids the expense of fine-tuning on components that gain more from context (procedures, state facts) while capturing weights gains where they exist

---

## Conclusion

Component routing demonstrates that **not all experience belongs in the weights**. By splitting trajectories into four components and routing each by two pre-measurable properties (recurrence $r$ and state-conditionality $\kappa$), the framework achieves consistent gains over whole-trajectory baselines and single-destination approaches across three backbone families and two environments.

Key takeaways:
1. One experience pool has **two destinations**: locators and lessons in weights; procedures and state facts in context
2. A simple rule in $(r, \kappa)$ recovers destinations for held-out backbones (24/24 cells) and survives interventions
3. Training makes notes redundant proportionally to recurrence; context gains track information gaps; weights gains suffer from policy gaps

**Future directions** include: applying component routing to other agent domains (web, robotics), dynamic routing where components migrate between destinations over the agent's lifetime, and extending the property-based framework beyond the two measured dimensions to capture additional aspects of experience value.

---

> *"Weights guide me to Search cities. On the preview, context says: tap Add. Same experience, two destinations. Right place, right time."* — Illustrative example of component routing in action

---

_Markdown view of https://picx.dev/p/gAnBHT, served by PicX — AI-generated visual whiteboard summaries of research papers._