# Learning from Fleet Experience: A Collective Learning Paradigm for Robots

> Collective Experience Learning lets robot fleets improve shared policies from deployment data alone, achieving 97.3% average success across five real-world manipulation tasks without expert corrections.

- **Source:** [arXiv](https://arxiv.org/abs/2610.25736)
- **Published:** 2026-10-21
- **Permalink:** https://picx.dev/p/c5wZPi
- **Whiteboard:** https://picx.dev/p/c5wZPi/image

## Summary

# Learning from Fleet Experience: A Collective Learning Paradigm for Robots

## Summary (Overview)

- **New learning paradigm**: The paper introduces **Collective Experience Learning (CEL)**, a paradigm in which deployed robots share execution experience to improve a shared policy without requiring additional expert demonstrations or corrective actions after initialization.

- **Key algorithmic contribution**: **Failure-guided Experience Retrieval fine-Tuning (FERT)** uses failure contexts from robot executions to retrieve relevant successful trajectory segments, using their actions as supervision for policy updates—failures guide selection, successes supply action targets.

- **System contribution**: A **Fault-tolerant Asynchronous Training System (FATS)** coordinates experience collection, policy optimization, and policy distribution across robot fleets despite local interruptions, with a data plane (SwarmLink) and control plane (HotSwarm).

- **Empirical results**: Across five real-world manipulation tasks, policies achieve **93.3–99.2% success rates** evaluated over 120 trials per task on held-out robots, with scaling improvements as fleet size increases (e.g., Phone Packaging improves from 79.2% to 98.3% with 96 robots).

- **Key finding**: The ablation study shows removing failure-guided retrieval reduces success from 86.7% to 64.2%, demonstrating that targeted experience selection is critical for effective learning from collective experience.

## Introduction and Theoretical Foundation

### Background and Motivation

Robot foundation models provide initial capabilities for physical action, but these initial capabilities do not automatically yield reliable execution due to variations in object properties, contact conditions, and individual robots. While robots operating in the real world generate valuable experience, the central constraint is the **human effort required to supply corrective actions** during deployment. As fleets grow, keeping teleoperators available to monitor execution and provide corrections becomes increasingly difficult.

### Problem Setting

The paper formalizes the objective as maximizing the probability of successful execution:

$$
J(\theta) = \mathbb{E}_{\tau \sim \pi_{\theta}}[y(\tau)]. \tag{1}
$$

where $\tau \sim \pi_{\theta}$ denotes a trajectory generated by executing the policy in the environment, $y(\tau) \in \{0,1\}$ indicates task success, and $\theta$ denotes policy parameters.

The policy is a vision-language-action model $\pi_{\theta}(A_t \mid o_t, \boldsymbol{g})$ where $A_t = (a_t, \ldots, a_{t+H-1})$ is an action chunk with prediction horizon $H$, $o_t$ is the robot observation, and $\boldsymbol{g}$ is the task instruction.

### Theoretical Foundation

The paper provides a theoretical analysis of FERT via three key results:

**Lemma 1 (Successful replay alignment)**: For current-policy successful replay, with $J(\theta_0) > 0$ and $\mathbf{v} \neq 0$:

$$
\mathbf{G}_{\mathrm{unif}} = \frac{\mathbf{v}}{T_c J(\theta_0)}, \qquad \mu = \frac{\|\mathbf{v}\|_2^2}{T_c J(\theta_0)} > 0, \tag{16}
$$

where $T_c$ is the episode length in action chunks. This shows fitting successful experience produces an update aligned with improving expected return.

**Lemma 2 (Retrieval alignment)**: Under a stage-wise robot task model with independent stages, if retrieval assigns at least as much weight to stages with higher local failure rates, then $\kappa > 0$, meaning successful experience from more failure-prone stages has greater first-order benefit.

**Theorem 1 (Local performance guarantee)**: Under the conditions of Lemmas 1 and 2, a sufficiently small positive step size gives:

$$
J(\theta_{\mathrm{FERT}}) > J(\theta_{\mathrm{unif}}) > J(\theta_0). \tag{18}
$$

with the lower bound:

$$
J(\theta_{\mathrm{FERT}}) - J(\theta_0) \geq \eta \underbrace{\mu}_{\text{successful replay}} + \eta \underbrace{\kappa}_{\text{retrieval contribution}} - \underbrace{\frac{\lambda_J \eta^2}{2}\|\mathbf{G}_{\mathrm{FERT}}\|_2^2}_{\text{finite-step penalty}}. \tag{19}
$$

## Methodology

### FERT Algorithm (Four Logical Steps)

**Step 1: Asynchronous experience collection**
- Successful trajectories are retained in a collective experience buffer $\mathcal{D}^+$
- Failed trajectories serve as temporary retrieval queries; their actions are not used as supervision
- Buffer insertion, retrieval, and priority updates occur between training updates

**Step 2: Experience encoding**

The visual encoder $f_{\bar{\theta}}$ produces a $16 \times 16$ grid of visual tokens per camera view, then applies spatial average pooling with a $4 \times 4$ kernel:

$$
h_t = \mathrm{Concat}_{c=0}^{C-1}\left[\mathrm{vec}\left(\mathrm{Pool}\left(f_{\bar{\theta}}(I_t^{(c)})\right)\right)\right]. \tag{2}
$$

The retrieval representation combines visual and proprioceptive features:

$$
z_t = \left[\frac{h_t}{\max(\|h_t\|_2, \epsilon)}; \frac{s_t}{\max(\|s_t\|_2, \epsilon)}\right]. \tag{3}
$$

**Step 3: Failure-guided retrieval**

Failure queries are constructed from observations preceding the endpoint of failed trajectories:

$$
e = T - \delta, \qquad b = \max(0, e - L), \qquad Q_\tau = (o_b, \ldots, o_{e-1}), \tag{4}
$$

with $L = 90$ and $\delta = 10$ frames (up to three seconds of context).

Retrieval uses **slope-constrained subsequence dynamic time warping (S-DTW)** with framewise distance matrix:

$$
d_{i,j}^{\tau_s} = \|z_i^{\tau_f} - z_j^{\tau_s}\|_2, \qquad 0 \leq i < M, \quad 0 \leq j < N_{\tau_s}. \tag{5}
$$

The alignment constraint allows successive matches to advance by one or two frames:

$$
0 \leq p_0, \quad p_{M-1} < N_{\tau_s}, \quad p_i - p_{i-1} \in \{1, 2\}, \quad i = 1, \ldots, M-1. \tag{6}
$$

**Step 4: Prioritized fine-tuning**

Priority updates for action chunks:

$$
w_i^{(n)} = \rho w_i^{(n-1)} + \sum_{R \in R_n} \frac{|\mathrm{idx}(A_i) \cap R|}{H}, \tag{10}
$$

Sampling is proportional to priorities:

$$
P(i) = \frac{w_i}{\sum_{j \in \mathcal{D}^+} w_j}, \qquad i \in \mathcal{D}^+. \tag{11}
$$

The training objective is:

$$
\mathcal{L}_{\mathrm{FERT}}(\theta) = -\sum_{i \in \mathcal{D}^+} \frac{w_i}{\sum_{j \in \mathcal{D}^+} w_j} \log \pi_{\theta}(A_i \mid o_i, g_i). \tag{13}
$$

### FATS System Architecture

FATS separates the learning loop into three worker types:
- **Environment workers**: operate robots (acquire observations, execute actions)
- **Inference workers**: compute actions from observations using the current policy
- **Training workers**: update the policy from collected experience

The system comprises two planes:
1. **SwarmLink data plane**: Distributed channels connecting robot execution to cloud inference and training, with trajectory streaming for cloud-side assembly
2. **HotSwarm control plane**: Manages station membership, worker identity, and tiered recovery (environment workers restart in place, inference workers reload policy, training workers resume from checkpoints)

## Empirical Validation / Results

### Main Results Across Five Tasks

The paper evaluates on five real-world manipulation tasks: Flower Arrangement, Clothes Folding, Phone Charging, Stool Assembly, and Phone Packaging.

**Table 3: Failure rate table**

| Task name | (M)HG-DAgger(%) ↓ | FERT(%) ↓ | Relative failure reduction (%) ↑ |
|-----------|-------------------|-----------|----------------------------------|
| Flower Arrangement | 1.67 | 0.83 | 50.0 |
| Clothes Folding | 14.17 | 1.67 | 88.2 |
| Phone Charging | 40.00 | 2.50 | 93.8 |
| Stool Assembly | 18.33 | 6.67 | 63.6 |
| Phone Packaging | 20.83 | 1.67 | 92.0 |
| **Task avg** | **19.00** | **2.67** | **86.0** |

Key findings:
- Multi-robot HG-DAgger raises mean success from 36.3% to 81.0%
- FERT further raises mean success to 97.3% (16.3 percentage points improvement)
- FERT exceeds human teleoperation on all five tasks (average advantage of 16.7 percentage points)
- FERT outperforms RLT and RECAP baselines by 12.3 and 11.5 percentage points on average

### Scaling with Fleet Size

- FERT success rates generally increase with fleet size across all five tasks
- On Phone Packaging: expanding from 16 to 96 robots raises success from 86.7% to 98.3% (11.6 percentage points gain)
- At smaller fleet sizes (8 robots), FERT can perform below its initialization, suggesting insufficiently rich successful experience

### Ablation Studies (Phone Packaging, 16 robots)

| Configuration | Success Rate |
|---------------|-------------|
| Full FERT (with FR) | 86.7% |
| Without failure-guided retrieval | 64.2% |
| Policy sync every 1,500 steps | 80.0% |
| Policy sync every 100 steps | 75.0% |
| No retrieved-segment extension | 57.5% |
| Priority retention ρ = 0.80 | 70.8% |
| Priority retention ρ = 0.95 | 82.5% |

### System Performance (96-robot run)

- Fleet collected **9,834 episodes** during the run (10:37–18:30)
- **30 fault incidents** on 22 of 96 robots; some robot interrupted every 22 min on average
- Incidents removed only **8.3 robot-hours** (1.3% of 660 robot-hours of operation)
- Synchronous collection would collect only 33% of episodes (vs. actual)
- Full-restart designs preserve 92%, 86%, and 74% of episodes for 2, 5, and 10 min restart durations
- 95.18% of observation channel operations finish within 20 ms; >99% of experience transfer operations finish within 2 minutes

## Theoretical and Practical Implications

### Theoretical Implications

The paper provides a formal framework for understanding when retrieval-weighted likelihood updates improve policy performance. The key insight is that **successful replay alone provides an improving update direction** (Lemma 1), and **retrieval adds a positive first-order contribution** when it prioritizes experience from more failure-prone stages (Lemma 2). The local performance guarantee (Theorem 1) decomposes the improvement into three terms: successful replay contribution, retrieval contribution, and finite-step penalty.

### Practical Implications

1. **Reduced human supervision burden**: CEL eliminates the need for expert corrective actions during deployment, requiring only binary outcome labels and environment resets. This makes scaling to large fleets more feasible.

2. **Fleet-size scalability**: The paradigm shows that larger fleets generally lead to better policies when collection time per robot is held fixed, suggesting a virtuous cycle where deployment expansion improves capabilities.

3. **Cross-robot transfer**: Policies trained on collective experience transfer to held-out robot instances, demonstrating that fleet-learned improvements generalize beyond contributing robots.

4. **System design principles**: The FATS architecture provides practical patterns for distributed robot learning: separating data and control planes, tiered recovery matching state rebuild costs, and asynchronous progress decoupling.

## Conclusion

This paper demonstrates that **experience generated during deployment can support further policy improvement** without requiring additional expert demonstrations or corrective actions. The CEL paradigm connects individual robot executions to learning across the fleet through a shared policy, offering a path toward improving robot reliability through collective operational experience.

Key takeaways:
- FERT effectively uses failures to guide experience selection while successes supply action targets
- FATS sustains the distributed learning loop despite local interruptions
- Results support scaling robot learning through fleet experience rather than expert supervision

**Future directions** identified by the authors include:
- Broader task distributions
- Heterogeneous robot platforms
- Adaptation to changes over longer periods of operation
- More broadly, connecting deployment expansion with a growing source of experience for shared learning, where fleets contribute both task execution and the experience from which their capabilities improve

---

_Markdown view of https://picx.dev/p/c5wZPi, served by PicX — AI-generated visual whiteboard summaries of research papers._