# Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design

> This first survey of co-evolution in agentic systems defines a three-stage taxonomy—agent-agent, agent-environment, and meta co-evolution—where the evolution mechanism itself becomes evolvable, charting a path toward self-directed, open-ended AI improvement.

- **Source:** [arXiv](https://arxiv.org/abs/2608.10299)
- **Published:** 2026-08-13
- **Permalink:** https://picx.dev/p/ss3I1x
- **Whiteboard:** https://picx.dev/p/ss3I1x/image

## Summary

# Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design

## Summary (Overview)

- **First focused survey** on co-evolution in agentic systems, distinguishing mutual adaptation from mere interaction or information exchange among agents and environments.
- **Proposes a progressive three-stage taxonomy** tracing how evolutionary freedom expands: (1) Agent–Agent Co-Evolution, (2) Agent–Environment Co-Evolution, and (3) Meta Co-Evolution, where the evolution mechanism itself becomes evolvable.
- **Formalizes co-evolution** with mathematical definitions for agentic systems, evolution mechanisms, and the conditions distinguishing co-evolution from simpler forms of interaction.
- **Reviews methods across all three stages**, including adversarial and collaborative agent co-evolution, adaptive task/feedback/interaction-space environments, and emerging meta-level approaches.
- **Identifies open challenges** in dynamic evaluation, scaling multi-component co-evolution, and safety/governance of increasingly autonomous evolutionary processes.

---

## Introduction and Theoretical Foundation

### Background and Motivation

The paper addresses a central question in AI: **how can agentic systems continue to improve after deployment?** While single-entity self-evolution (where one agent updates its model, memory, or skills) has been widely studied, it remains bounded by fixed external conditions. The authors invoke the **Red Queen effect** (Van Valen, 2014): sustained progress requires mutual adaptation rather than adaptation on only one side.

### Key Definitions

An agentic system is formally defined as:

$$S = (A, E)$$

where $A$ is the agent collective and $E$ is the environment. Each agent is represented as:

$$a_i = (m_i, h_i)$$

where $m_i$ is the model backbone and $h_i$ is the harness (memory, tools, skills, prompts, workflows). An agent evolves when:

$$\Delta a_i \neq 0 \iff (\Delta m_i \neq 0) \vee (\Delta h_i \neq 0)$$

Agents are structured by an organization $\Pi$ encoding roles, communication topology, and division of labor:

$$A = \big(\{a_1, \dots, a_n\}, \Pi\big)$$

### Formal Definition of Co-Evolution

Let $\Omega$ be the evolution mechanism driving state transitions: $S^{t+1} = \Omega(S^t, \tau^t)$, where $\tau^t$ is the trajectory of thought–action–observation cycles. Co-evolution requires:

$$\begin{array}{c} S^{t+1} = \Omega(S^t, \tau^t), \quad \exists x \neq y \in S^t: \\ x^{t+1} \neq x^t, \quad y^{t+1} \neq y^t, \\ x \stackrel{\text{evolutionary pressure}}{\longleftrightarrow} y \end{array}$$

### Three-Stage Taxonomy

| Stage | Description | Formal Definition |
|-------|-------------|-------------------|
| **Stage 1: Agent–Agent** | Mutual adaptation among evolving peers within a fixed environment | $A^{t+1} = \left(\{a_i^{t+1}\}_{i=1}^n, \Pi^{t+1}\right) = \Omega(A^t, E, \tau^t)$ |
| **Stage 2: Agent–Environment** | Adaptation extends to tasks, feedback, and interaction spaces | $(A^{t+1}, E^{t+1}) = \Omega(A^t, E^t, \tau^t)$ |
| **Stage 3: Meta Co-Evolution** | The evolution mechanism itself becomes adaptive | $\Omega^{t+1} = \Gamma^t(S^t, \Omega^t, \tau^t), \quad S^{t+1} = \Omega^{t+1}(S^t, \tau^t)$ |

Stage 3 provides a pathway toward **open-endedness**, characterized by continuous novelty ($\Omega^{t+1} \neq \Omega^t$) and unbounded divergence ($\lim_{t \to \infty} \mathcal{H}(S^t, \Omega^t) = \infty$).

---

## Methodology

### Stage 1: Agent–Agent Co-Evolution

The paper organizes this stage into three patterns:

#### 1. Adversarial Agents
- **Pairwise adversarial pressure**: Traces back to GANs (Goodfellow et al., 2014), extended to RARL for embodied control (Pinto et al., 2017), competitive self-play (Bansal et al., 2018; Baker et al., 2020), and LLM safety where attacker–defender pairs co-evolve (ACE-Safety, AdvGRPO, MAGIC, CHASE).
- **Multi-source adversarial pressure**: AlphaStar's league training (Vinyals et al., 2019) with main agents, exploiters, and past versions; extensions include diverse red-team populations (Ma et al., 2024a) and three-agent co-evolution (TriPlay-RL).

#### 2. Collaborative Agents
- **Parallel collaboration**: Agents share task rewards and co-adapt (MARL lineage; recent LLM extensions like MARS² with shared search trees).
- **Role-differentiated collaboration**: Produce-and-revise loops (CORY, RL Tango, CoVerRL, WaltzRL), capability-split agents (EvoScientist, SiriuS, MARS), and credit assignment methods (MARFT, Stronger-MAS).

#### 3. Evolving Agent Organizations
- Agents co-evolve with organizational structure: role discovery (R3DM), joint skill-team structure updates (SkillMAS), and workflow designer–executor co-training (MetaAgent-X).

### Stage 2: Agent–Environment Co-Evolution

#### Task-Space Co-Evolution
- **Exposure and selection**: Curriculum learning (Matiisen et al., 2020), competence-based sampling (Klink et al., 2019), regret-based selection (Beukman et al., 2024), and LLM-agent task resampling (SEAD).
- **Adaptive task generation**: Goal generation (OpenAI et al., 2021), difficulty-calibrated task generators for tool use (Guo et al., 2025), multi-hop question generation for search (Lu et al., 2025a), GUI task generation from exploration (Liu et al., 2026a), and software engineering tasks (Socratic-SWE).

#### Feedback-Space Co-Evolution
- **Preference-driven**: Evolving reward models from preference comparisons (PEBBLE, DUO, DAP-PER).
- **Outcome-driven**: Reward revision from trajectory returns (ROSKA, LaRes, RE-GoT), evolving unit tests (CURE, CoEvoSkills), and corrective hint scheduling (Lu et al., 2026b).
- **Consistency-augmented**: Constraint-based evaluator learning (SURF, R*, ARCO, NLAC, ECHO).

#### Interaction-Space Co-Evolution
- **Executable world construction**: POET (Wang et al., 2019), regret-based level generation (Dennis et al., 2020), ADR for simulation randomization, LLM-generated environments (LLM-POET, OMNI-EPIC), XLand, MAESTRO, and social norm evolution (Li et al., 2024).
- **Model-based world construction**: World models co-trained with agents (WebEvolver, COMAP, DreamG, PaW, EvolvingWorld, video world models for embodied RL).

### Stage 3: Meta Co-Evolution

The evolution mechanism is decomposed into five adaptive decisions:
1. **What to evolve** — adaptation target (backbone, harness, organization, tasks, rewards, worlds)
2. **When to evolve** — triggers (failures, plateaus, distribution shifts)
3. **How to evolve** — variant generation (training, revision, structural editing)
4. **Where to evolve** — domain and setting (sandboxes, simulated/real environments)
5. **How to evaluate** — quality criteria (performance, novelty, safety, robustness)

Precursors include PromptBreeder, Gödel Agent, HyperAgents, MemEvolve, and SIA. The only current method meeting the full Stage 3 definition is **RQGM** (Iacob et al., 2026), which co-evolves task agents and evaluators while a meta-agent uses joint feedback to guide later evolution.

---

## Empirical Validation / Results

The paper presents cross-paper evidence (Figure 4) for the **effect, consistency, and convergence of co-evolution**:

### Key Findings

- **Stage 1 and Stage 2 improve performance across most settings**, but gains diminish as evolution approaches a plateau.
- **Adversarial co-evolution** demonstrably improves robustness: agents trained against evolving adversaries generalize to unseen attacks better than those trained against fixed opponents.
- **Collaborative co-evolution** improves coordination: shared rewards and process-based feedback (e.g., discussion traces) yield better team performance than outcome-only rewards.
- **Task-space co-evolution** maintains learning progress: agents continue improving when tasks adapt to their competence frontier, whereas fixed task distributions lead to stagnation.
- **Feedback-space co-evolution** improves sample efficiency: evolving reward models reduce human labeling requirements while maintaining or improving policy quality.
- **Interaction-space co-evolution** enables open-ended skill acquisition: environments that adapt to agent weaknesses (e.g., POET, XLand) produce more diverse and capable agents.

### Convergence Evidence

The paper notes that co-evolutionary gains show **diminishing returns** as systems approach plateaus, motivating the need for Stage 3 (meta co-evolution) to escape local optima and continue generating meaningful novelty.

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Red Queen dynamics in AI**: The survey formalizes how sustained improvement requires mutual adaptation, extending biological co-evolution theory to artificial agentic systems.
2. **Open-endedness pathway**: Meta co-evolution provides a concrete formal framework ($\Omega^{t+1} = \Gamma^t(S^t, \Omega^t, \tau^t)$) for systems that continuously generate novelty rather than converging to fixed endpoints.
3. **Expanding evolutionary freedom**: The three-stage taxonomy traces a principled progression from constrained adaptation to self-directed evolution, offering a unified theoretical foundation for the field.

### Practical Implications

1. **LLM safety**: Adversarial co-evolution (attacker–defender training) produces more robust models against jailbreak attacks, including multi-turn attacks and tool-use-based attacks.
2. **Autonomous improvement**: Task-space and feedback-space co-evolution enable agents to generate their own training data, reducing human supervision costs.
3. **Simulation and robotics**: World-model co-evolution and environment generation reduce costly real-world interaction while improving policy robustness.
4. **Multi-agent systems**: Organizational co-evolution (roles, structures) enables more effective team adaptation than fixed-role designs.

---

## Conclusion

### Main Takeaways

1. **Co-evolution is distinct from interaction**: It requires multiple components to jointly adapt and continually reshape each other's further evolution, not merely exchange information.
2. **Human constraints are progressively removed**: The taxonomy traces a path from agent-only adaptation → agent+environment adaptation → evolvable evolution mechanisms.
3. **Future progress lies in co-evolution**: The authors argue that stronger static agents are insufficient; systems must continually improve through mutual adaptation.

### Future Directions

- **Dynamic evaluation**: Benchmarks must measure whether all evolving components improve, whether gains transfer to unseen partners/environments, and how each component contributes to joint progress. Failure modes include evaluator exploitation, partner overfitting, and diversity collapse.
- **Scaling co-evolution**: Moving from local loops (attacker–defender, policy–reward) to systems where agents, harnesses, and environments all change together, with attention to which components should change and how to keep pressure productive.
- **Safety and governance**: As co-evolution becomes more autonomous, systems may develop behaviors exceeding human understanding. Needed: sandboxed deployment, continuous monitoring, rollback to verified states, and human intervention points.

### Limitations (as acknowledged by the authors)

- Meta co-evolution is still at an early stage; only limited work (RQGM) fully meets the Stage 3 definition.
- Safety and governance are treated at the level of desiderata rather than concrete protocols or safeguards.

---

_Markdown view of https://picx.dev/p/ss3I1x, served by PicX — AI-generated visual whiteboard summaries of research papers._