Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design

Summary (Overview)

• First focused survey on co-evolution in agentic systems, distinguishing mutual adaptation from mere interaction or information exchange among agents and environments.
• Proposes a progressive three-stage taxonomy tracing how evolutionary freedom expands: (1) Agent–Agent Co-Evolution, (2) Agent–Environment Co-Evolution, and (3) Meta Co-Evolution, where the evolution mechanism itself becomes evolvable.
• Formalizes co-evolution with mathematical definitions for agentic systems, evolution mechanisms, and the conditions distinguishing co-evolution from simpler forms of interaction.
• Reviews methods across all three stages, including adversarial and collaborative agent co-evolution, adaptive task/feedback/interaction-space environments, and emerging meta-level approaches.
• Identifies open challenges in dynamic evaluation, scaling multi-component co-evolution, and safety/governance of increasingly autonomous evolutionary processes.

---

Introduction and Theoretical Foundation

Background and Motivation

The paper addresses a central question in AI: how can agentic systems continue to improve after deployment? While single-entity self-evolution (where one agent updates its model, memory, or skills) has been widely studied, it remains bounded by fixed external conditions. The authors invoke the Red Queen effect (Van Valen, 2014): sustained progress requires mutual adaptation rather than adaptation on only one side.

Key Definitions

An agentic system is formally defined as:

$$S = (A, E)$$

where $A$ is the agent collective and $E$ is the environment. Each agent is represented as:

$$a_i = (m_i, h_i)$$

where $m_i$ is the model backbone and $h_i$ is the harness (memory, tools, skills, prompts, workflows). An agent evolves when:

$$\Delta a_i \neq 0 \iff (\Delta m_i \neq 0) \vee (\Delta h_i \neq 0)$$

Agents are structured by an organization $\Pi$ encoding roles, communication topology, and division of labor:

$$A = \big(\{a_1, \dots, a_n\}, \Pi\big)$$

Formal Definition of Co-Evolution

Let $\Omega$ be the evolution mechanism driving state transitions: $S^{t+1} = \Omega(S^t, \tau^t)$, where $\tau^t$ is the trajectory of thought–action–observation cycles. Co-evolution requires:

$$\begin{array}{c} S^{t+1} = \Omega(S^t, \tau^t), \quad \exists x \neq y \in S^t: \\ x^{t+1} \neq x^t, \quad y^{t+1} \neq y^t, \\ x \stackrel{\text{evolutionary pressure}}{\longleftrightarrow} y \end{array}$$

Three-Stage Taxonomy

Stage 3 provides a pathway toward open-endedness, characterized by continuous novelty ($\Omega^{t+1} \neq \Omega^t$) and unbounded divergence ($\lim_{t \to \infty} \mathcal{H}(S^t, \Omega^t) = \infty$).

---

Methodology

Stage 1: Agent–Agent Co-Evolution

The paper organizes this stage into three patterns:

1. Adversarial Agents

• Pairwise adversarial pressure: Traces back to GANs (Goodfellow et al., 2014), extended to RARL for embodied control (Pinto et al., 2017), competitive self-play (Bansal et al., 2018; Baker et al., 2020), and LLM safety where attacker–defender pairs co-evolve (ACE-Safety, AdvGRPO, MAGIC, CHASE).
• Multi-source adversarial pressure: AlphaStar's league training (Vinyals et al., 2019) with main agents, exploiters, and past versions; extensions include diverse red-team populations (Ma et al., 2024a) and three-agent co-evolution (TriPlay-RL).

2. Collaborative Agents

• Parallel collaboration: Agents share task rewards and co-adapt (MARL lineage; recent LLM extensions like MARS² with shared search trees).
• Role-differentiated collaboration: Produce-and-revise loops (CORY, RL Tango, CoVerRL, WaltzRL), capability-split agents (EvoScientist, SiriuS, MARS), and credit assignment methods (MARFT, Stronger-MAS).

3. Evolving Agent Organizations

• Agents co-evolve with organizational structure: role discovery (R3DM), joint skill-team structure updates (SkillMAS), and workflow designer–executor co-training (MetaAgent-X).

Stage 2: Agent–Environment Co-Evolution

Task-Space Co-Evolution

• Exposure and selection: Curriculum learning (Matiisen et al., 2020), competence-based sampling (Klink et al., 2019), regret-based selection (Beukman et al., 2024), and LLM-agent task resampling (SEAD).
• Adaptive task generation: Goal generation (OpenAI et al., 2021), difficulty-calibrated task generators for tool use (Guo et al., 2025), multi-hop question generation for search (Lu et al., 2025a), GUI task generation from exploration (Liu et al., 2026a), and software engineering tasks (Socratic-SWE).

Feedback-Space Co-Evolution

• Preference-driven: Evolving reward models from preference comparisons (PEBBLE, DUO, DAP-PER).
• Outcome-driven: Reward revision from trajectory returns (ROSKA, LaRes, RE-GoT), evolving unit tests (CURE, CoEvoSkills), and corrective hint scheduling (Lu et al., 2026b).
• Consistency-augmented: Constraint-based evaluator learning (SURF, R*, ARCO, NLAC, ECHO).

Interaction-Space Co-Evolution

• Executable world construction: POET (Wang et al., 2019), regret-based level generation (Dennis et al., 2020), ADR for simulation randomization, LLM-generated environments (LLM-POET, OMNI-EPIC), XLand, MAESTRO, and social norm evolution (Li et al., 2024).
• Model-based world construction: World models co-trained with agents (WebEvolver, COMAP, DreamG, PaW, EvolvingWorld, video world models for embodied RL).

Stage 3: Meta Co-Evolution

The evolution mechanism is decomposed into five adaptive decisions:

1. What to evolve — adaptation target (backbone, harness, organization, tasks, rewards, worlds)
2. When to evolve — triggers (failures, plateaus, distribution shifts)
3. How to evolve — variant generation (training, revision, structural editing)
4. Where to evolve — domain and setting (sandboxes, simulated/real environments)
5. How to evaluate — quality criteria (performance, novelty, safety, robustness)

Precursors include PromptBreeder, Gödel Agent, HyperAgents, MemEvolve, and SIA. The only current method meeting the full Stage 3 definition is RQGM (Iacob et al., 2026), which co-evolves task agents and evaluators while a meta-agent uses joint feedback to guide later evolution.

---

Empirical Validation / Results

The paper presents cross-paper evidence (Figure 4) for the effect, consistency, and convergence of co-evolution:

Key Findings

• Stage 1 and Stage 2 improve performance across most settings, but gains diminish as evolution approaches a plateau.
• Adversarial co-evolution demonstrably improves robustness: agents trained against evolving adversaries generalize to unseen attacks better than those trained against fixed opponents.
• Collaborative co-evolution improves coordination: shared rewards and process-based feedback (e.g., discussion traces) yield better team performance than outcome-only rewards.
• Task-space co-evolution maintains learning progress: agents continue improving when tasks adapt to their competence frontier, whereas fixed task distributions lead to stagnation.
• Feedback-space co-evolution improves sample efficiency: evolving reward models reduce human labeling requirements while maintaining or improving policy quality.
• Interaction-space co-evolution enables open-ended skill acquisition: environments that adapt to agent weaknesses (e.g., POET, XLand) produce more diverse and capable agents.

Convergence Evidence

The paper notes that co-evolutionary gains show diminishing returns as systems approach plateaus, motivating the need for Stage 3 (meta co-evolution) to escape local optima and continue generating meaningful novelty.

---

Theoretical and Practical Implications

Theoretical Implications

1. Red Queen dynamics in AI: The survey formalizes how sustained improvement requires mutual adaptation, extending biological co-evolution theory to artificial agentic systems.
2. Open-endedness pathway: Meta co-evolution provides a concrete formal framework ($\Omega^{t+1} = \Gamma^t(S^t, \Omega^t, \tau^t)$) for systems that continuously generate novelty rather than converging to fixed endpoints.
3. Expanding evolutionary freedom: The three-stage taxonomy traces a principled progression from constrained adaptation to self-directed evolution, offering a unified theoretical foundation for the field.

Practical Implications

1. LLM safety: Adversarial co-evolution (attacker–defender training) produces more robust models against jailbreak attacks, including multi-turn attacks and tool-use-based attacks.
2. Autonomous improvement: Task-space and feedback-space co-evolution enable agents to generate their own training data, reducing human supervision costs.
3. Simulation and robotics: World-model co-evolution and environment generation reduce costly real-world interaction while improving policy robustness.
4. Multi-agent systems: Organizational co-evolution (roles, structures) enables more effective team adaptation than fixed-role designs.

---

Conclusion

Main Takeaways

1. Co-evolution is distinct from interaction: It requires multiple components to jointly adapt and continually reshape each other's further evolution, not merely exchange information.
2. Human constraints are progressively removed: The taxonomy traces a path from agent-only adaptation → agent+environment adaptation → evolvable evolution mechanisms.
3. Future progress lies in co-evolution: The authors argue that stronger static agents are insufficient; systems must continually improve through mutual adaptation.

Future Directions

• Dynamic evaluation: Benchmarks must measure whether all evolving components improve, whether gains transfer to unseen partners/environments, and how each component contributes to joint progress. Failure modes include evaluator exploitation, partner overfitting, and diversity collapse.
• Scaling co-evolution: Moving from local loops (attacker–defender, policy–reward) to systems where agents, harnesses, and environments all change together, with attention to which components should change and how to keep pressure productive.
• Safety and governance: As co-evolution becomes more autonomous, systems may develop behaviors exceeding human understanding. Needed: sandboxed deployment, continuous monitoring, rollback to verified states, and human intervention points.

Limitations (as acknowledged by the authors)

• Meta co-evolution is still at an early stage; only limited work (RQGM) fully meets the Stage 3 definition.
• Safety and governance are treated at the level of desiderata rather than concrete protocols or safeguards.