# Are Self-Evolving Agents Continual Learners?

> Self-evolving LLM agents trade stability for plasticity, failing as continual learners because global prompt edits cause cross-task interference and similarity-based retrieval collides on conflicting workflows.

- **Source:** [arXiv](https://arxiv.org/abs/2610.25858)
- **Published:** 2026-10-24
- **Permalink:** https://picx.dev/p/W4SgHu
- **Whiteboard:** https://picx.dev/p/W4SgHu/image

## Summary

# Are Self-Evolving Agents Continual Learners?

**Authors:** Xiaoyin Chen, Yizheng Huang, Yinglun Xu, Ziyun Zhao, Emmett McQuinn, Julie Liu, Hope Yao (Mila – Quebec AI Institute, Université de Montréal, Google DeepMind)

---

## Summary (Overview)

- **Core question**: Are nonparametric self-evolving LLM agents (which adapt by rewriting prompts, memory, and skills while keeping weights frozen) effective continual learners on drifting task streams?
- **Key finding**: Self-evolution improves aggregate performance in most settings, but **higher plasticity consistently comes at the cost of stability** — no method achieves both simultaneously.
- **Three representative methods evaluated**: ReasoningBank (experience retrieval), GEPA (prompt self-modification), and Meta-Harness (full workspace modification) across four drift scenarios (topic, workflow, domain, compound).
- **Two failure mechanisms identified**: (1) Self-modifying methods apply global edits to shared prompts/workspaces, causing cross-task interference and overwriting of earlier knowledge; (2) Similarity-based retrieval injects foreign memories and suffers from retrieval collision when similar queries require conflicting workflows.
- **Conclusion**: Current self-evolving agents remain **incomplete continual learners**, requiring new mechanisms for isolating task-specific updates and conditioning retrieval beyond surface query similarity.

---

## Introduction and Theoretical Foundation

### Background and Motivation

LLM agents are increasingly deployed in environments where **new tasks, tools, and workflows continuously emerge** after model weights are frozen. An agent serving users over extended periods must:

- Learn each new task as it arrives
- Preserve previously acquired knowledge

This resembles the classical **continual learning** problem, but at a **pace that conventional gradient updates cannot sustain** — retraining after every session is prohibitively expensive, and fine-tuning on recent interactions erodes general capabilities.

### The Nonparametric Alternative

The **in-context learning ability** of LLMs suggests an alternative: instead of updating weights, the agent updates its **harness** (system prompt, memory, skill libraries, control code). Two paradigms have emerged:

1. **Self-modification**: Rewrites the artifacts the agent runs on (system prompt, harness code) — e.g., GEPA, Meta-Harness
2. **Experience augmentation**: Appends reflections/trajectories to an external store, retrieved during execution — e.g., ReasoningBank

Both paradigms show gains on **stationary, single-domain benchmarks**, but their continual learning behavior remains unexamined.

### Theoretical Formulation

A **task family** is defined as:

$$
\mathcal{F} = (\mathcal{X}, \mathcal{D}, \mathcal{Y}, \mathcal{R})
$$

where $\mathcal{X}$ is the input space, $\mathcal{D}$ is the input distribution, $\mathcal{Y}$ is the output space, and $R: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ is a reward function. In the agentic setting, episodes are trajectories and rewards are binary task success: $R(x,y) \in \{0,1\}$.

A **task stream** over $K$ families with weight schedule $w = (w_t)_{t=0}^{T-1}$ draws inputs from the mixture:

$$
x_t \sim P_t := \sum_{k=1}^{K} w_{t,k} \mathcal{D}_k, \qquad t \in \{0, \dots, T-1\}. \tag{1}
$$

The **continual learning objective** maximizes expected average performance over the stream:

$$
\bar{J}_w := \mathbb{E}_{S_w} \bigg[ \frac{1}{T} \sum_{t=1}^{T} J\big(\pi^{(t)}\big) \bigg], \tag{3}
$$

where $J(\pi) = \frac{1}{K} \sum_{k=1}^{K} J_k(\pi)$ evaluates performance on **all** $K$ families — creating the fundamental tension between plasticity and forgetting.

### The Evolver

A self-evolving agent updates its policy through an **evolver** $f$:

$$
c_{t+1} = f\big(c_t, \mathcal{H}_t\big), \tag{5}
$$

where $c_t$ is the harness at time $t$ and $\mathcal{H}_t$ is the accumulated experience history. The evolver's performance under schedule $w$ is:

$$
\bar{J}_w(f) := \mathbb{E}_{S_{w,f}} \bigg[ \frac{1}{T} \sum_{t=1}^{T} J\big(\pi^{(t)}\big) \bigg]. \tag{6}
$$

---

## Methodology

### Task Stream Configurations

Four **drifting stream** scenarios were constructed, each with one-hot, piecewise-constant weight schedules (contiguous blocks of 50 training tasks per group):

| Stream | $T$ | $K$ | Task groups (in order) | Shift type |
|--------|-----|-----|----------------------|------------|
| **Topic drift** | 150 | 3 | BIRD-F1 → BIRD-Code → BIRD-TP | Same environment/workflow, changing topic |
| **Workflow drift** | 100 | 2 | BFCL-Miss → BFCL-Base | Same APIs, conflicting workflows |
| **Domain drift** | 150 | 3 | ScienceWorld → BFCL-Base → BIRD-F1 | Changing environment and action space |
| **Compound drift** | 200 | 4 | BIRD-Code → BIRD-F1 → BFCL-Miss → BFCL-Base | Sequential combination of all shifts |

### Evaluation Metrics

**Seed baseline**: Per-family accuracy $B_k := A[0,k]$ from the initial harness before any evolution.

**Overall performance** (offset against seed, in percentage points):

$$
\bar{\Delta} = \frac{1}{MK} \sum_{m=1}^{M} \sum_{k=1}^{K} \big(A[t_m,k] - B_k\big), \quad \text{Term.} = \frac{1}{K} \sum_{k=1}^{K} \big(A[T,k] - B_k\big). \tag{8}
$$

**Plasticity** $P$ — learning during a family's own block:

$$
P = \frac{1}{K} \sum_{k=1}^{K} \big(A[\tau_k, k] - B_k\big). \tag{9}
$$

**Stability** $S$ — combined pre-block interference and post-block forgetting:

$$
I_k = \frac{1}{M_k} \sum_{0 < t_m \leq \sigma_k} \max\big(0, B_k - A[t_m,k]\big), \quad F_k = \max\big(0, A[\tau_k,k] - A[T,k]\big), \tag{10}
$$

$$
S = -\frac{1}{K} \sum_{k=1}^{K} (I_k + F_k) = -(\bar{I} + \bar{F}). \tag{11}
$$

### Experimental Setup

- **Base model**: Gemini 3.6 Flash in a ReAct-style loop
- **Evaluation**: Checkpoints at every block boundary (every 50 positions), 8 rollouts per instance
- **Methods compared**:
  - **ReasoningBank**: Distills episodes into memory items, retrieves top-1 most similar item
  - **GEPA**: Rewrites a shared strategy section in the system prompt via reflective mutation
  - **Meta-Harness**: Proposes edits to full workspace (prompt, memory, skills, controller code) via coding agent with validation gate

---

## Empirical Validation / Results

### 1. Current Methods Trade Stability for Plasticity

**Table 1** (abbreviated) | Continual learning performance across drift scenarios (pp. over seed):

| Stream | Seed | ReasoningBank Term./P/S | GEPA Term./P/S | Meta-Harness Term./P/S |
|--------|------|------------------------|----------------|------------------------|
| Topic | 52.5 | +0.3 / +0.7 / −1.1 | −1.7 / +1.7 / −4.8 | −1.5 / −2.0 / −0.3 |
| Workflow | 65.5 | +4.5 / +8.5 / −4.8 | +4.5 / +11.0 / −6.5 | +4.8 / +3.8 / −0.5 |
| Domain | 69.6 | +2.6 / +1.1 / −0.4 | +6.2 / +4.7 / −2.4 | +1.6 / +4.2 / −3.1 |
| Compound | 60.4 | +2.8 / +3.1 / −1.4 | +5.8 / +7.5 / −3.3 | +0.8 / +2.9 / −4.4 |
| **Overall** | **62.0** | **+2.5 / +3.3 / −1.9** | **+3.7 / +6.2 / −4.3** | **+1.4 / +2.2 / −2.1** |

**Key patterns**:
- **GEPA** has highest plasticity (+6.2 avg) but largest stability loss (−4.3 avg) — rewriting the shared prompt gives broad behavioral change at the cost of cross-task interference
- **ReasoningBank** achieves best stability (−1.9 avg) with roughly half of GEPA's plasticity — localization via retrieval constrains interference
- **Meta-Harness** is least stable on domain/compound drift; its apparent stability on topic/workflow comes from its validation gate **rejecting half or more candidate updates** (freezing the workspace at the cost of low plasticity)

A consistent **negative correlation** between plasticity and stability emerges across all scenarios and methods ($n=12$).

### 2. Self-Modification Causes Cross-Task Interference and Overwriting

**Pre-block interference**: Rules learned on earlier blocks apply unconditionally to unseen tasks:
- Domain drift: ScienceWorld training lowers BFCL-Base by 5.5 (GEPA), 1.5 (Meta-Harness) before BFCL tasks arrive
- Compound drift: BIRD blocks degrade BFCL-Miss to −6.8 (GEPA) before any BFCL task arrives
- Evolved prompts embed SQLite exploration instructions that cause the agent to invoke lookup tools on BFCL tasks instead of asking for missing arguments

**Post-block forgetting** via compression and overwriting (Figure 3a,b):
- Topic drift: BIRD-F1's prompt share falls from 100% → 75.2% → 40.2% as later blocks write over it; F1 accuracy drops from +7.5 → +3.5 → −2.0
- Compound drift: Seven BIRD SQL sections (3,386 chars) compressed to a single 1,034-char section; BIRD-F1's share drops from 59.5% → 5.8%, erasing its +4.0 gain

**Behavioral reversal under conflicting workflows** (Figure 3c): With BFCL-Miss and BFCL-Base sharing nearly identical queries but requiring opposite workflows:
- After BFCL-Miss training: clarification rate rises to 29.8% (GEPA) / 60.8% (Meta-Harness)
- After BFCL-Base rewrites instructions: clarification rate drops to 11.2% / 8.5%, causing BFCL-Miss accuracy drops of 13.0 / 10.2 points

### 3. Similarity-Based Retrieval Cannot Prevent Interference or Collision

**Domain separation prevents forgetting**: When domains are well-separated, cross-group nearest-match cosine similarity (0.57–0.63) is much lower than within-group (0.80–0.95), yielding $\bar{F} = 0.0$ on domain drift.

**However**, retrieval still fails:

- **Pre-block interference**: Before a group's block, 100% of retrievals are foreign memories (zero in-group entries exist), pulling accuracy below seed (e.g., BFCL-Miss −3.2 on compound drift)
- **Query similarity is unreliable**: Same-task BIRD queries across topics have similarity (0.60–0.64) barely above cross-domain queries (0.56–0.63)
- **Retrieval collision**: BFCL-Miss and BFCL-Base queries have cross-group nearest-match similarity (0.835–0.840) **indistinguishable from within-group** (0.834–0.837). At stream end, 52% of BFCL-Miss episodes retrieve BFCL-Base memories (and 46% vice versa), causing accuracy drops of 8.0 points

---

## Theoretical and Practical Implications

### For Agent Design

1. **Harness update scope is a plasticity-stability dial**: Expanding the editable surface (memory → prompt → full workspace) increases plasticity but proportionally reduces stability. Designers must consciously choose where on this trade-off their deployment lives.

2. **Global edits are fundamentally problematic**: Any mechanism that writes task-specific rules to a shared surface applied to all rollouts will cause interference on unseen tasks and overwriting of earlier knowledge.

3. **Similarity is not compatibility**: Retrieval collision shows that embedding-space similarity captures topical overlap but cannot determine whether a stored trajectory's *workflow* is compatible with the incoming task.

### Required Advancements

The authors' findings call for:
- **Mechanisms that isolate task-specific updates** in the harness (e.g., modular or scoped prompt sections with explicit task-group identification)
- **Conditioned retrieval beyond surface similarity** (e.g., workflow-aware or verification-based retrieval that can reject incompatible memories)
- **Explicit detection of distribution shift** at block boundaries — the agent currently must infer stream structure on its own, which fails when domains look similar but require different behaviors

### Broader Significance

This work bridges the **continual learning** and **self-evolving agent** literatures, formalizing the first framework for evaluating harness-level adaptation under non-stationary task streams, and revealing that the "free" adaptation of frozen-weight agents still exhibits the same plasticity-stability dilemma as parametric continual learning.

---

## Conclusion

**Main takeaway**: Current nonparametric self-evolving agents are **incomplete continual learners**.

- Self-modifying methods (GEPA, Meta-Harness) achieve plasticity through global rewrites, but cause pre-block interference and post-block forgetting — most severely when consecutive blocks share a domain but require conflicting workflows
- Experience augmentation (ReasoningBank) prevents overwriting across distant domains but injects foreign memories before a task group is encountered and suffers retrieval collision when distinct workflows share the embedding space
- No current method achieves both high plasticity and high stability on any drift scenario

**Future directions**: The authors call for harness architectures that support **task-isolated updates** and **workflow-aware retrieval**, moving beyond similarity-based matching to condition on whether a stored trajectory's behavioral requirements are compatible with the incoming task.

---

_Markdown view of https://picx.dev/p/W4SgHu, served by PicX — AI-generated visual whiteboard summaries of research papers._