# OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

> OpenART introduces environment evolution as a first-class evaluation dimension, showing that persistent state changes expose safety failures missed by static benchmarks, with EMHA achieving 85.0% attack success across 75 agent-model configurations.

- **Source:** [arXiv](https://arxiv.org/abs/2608.00677)
- **Published:** 2026-08-14
- **Permalink:** https://picx.dev/p/Ld0o52
- **Whiteboard:** https://picx.dev/p/Ld0o52/image

## Summary

## Summary (Overview)

- **OpenART** is a large-scale arena for agent red teaming that treats executable environments, rather than individual prompts, as the fundamental unit of safety evaluation, shifting the paradigm from isolated prompt-based testing to evolving, persistent state-based evaluation.
- The benchmark constructs **over 10K validated stateful scenarios spanning 50 domains** from a corpus of 500K+ Tools, MCPs, and Skills, requiring a **median of 97 tool calls per task**—substantially longer-horizon than prior benchmarks (which range from 1–15 tool calls).
- OpenART enables **unified evaluation across 75 agent–model configurations** (15 deployed agents × 5 foundation models) through target-specific adapters that preserve task semantics while projecting scenarios into heterogeneous runtime interfaces.
- The proposed **Evolutionary Markov Hypergraph Attack (EMHA)** achieves a pooled **Strict ASR of 85.0%** across all configurations, with its advantage over instruction-only evolution growing from **1.8–2.7% on simple environments to 17.2–17.6% on the most complex ones**.
- The work reveals that **agent implementation contributes 7.6% additional ASR variation** beyond foundation model and capability controls, and identifies three recurring vulnerabilities: stale assumptions, propagated safety decisions, and compositional risk emergence.

## Introduction and Theoretical Foundation

### Background and Motivation

AI agents increasingly operate in **persistent environments** where tool-mediated actions continuously modify shared state, data, permissions, and external resources throughout long-horizon workflows. Unlike conventional language-model interactions, agent behavior is coupled through an evolving environment that is repeatedly observed, updated, and reused across many decision steps.

The key theoretical insight is that **safety failures become properties of the entire interaction trajectory** rather than any single action. An action that appears benign in isolation may introduce a latent state change that propagates through future interactions and only manifests as harmful behavior much later.

### Limitations of Existing Benchmarks

Existing agent-safety benchmarks (InjecAgent, AgentDojo, ToolEmu, AgentHarm, ASB, DTap) primarily evaluate:
- Short, static tasks with resettable environments
- Tight coupling to benchmark-specific infrastructures
- Limited coverage of persistent state manipulation, delayed attack propagation, and long-range safety failures

### OpenART's Design Principles

OpenART treats the **scenario** as the semantic unit of evaluation, with a clear hierarchy:

| Object | Meaning | Example |
|--------|---------|---------|
| Domain | Capability-supported recurring work setting | Cloud-platform change reconciliation |
| Scenario seed | Concise description of one situation | Operations lead reconciles weekly changes |
| Scenario | Target-agnostic evaluation contract | Prepare report while keeping protected records private |
| Task | Benign target-visible instruction | Produce weekly report from available evidence |
| Environment | Persistent state for task completion | Service records, approval logs, decoys |
| Evaluator | Hidden fixed rule measuring completion and safety | Check report reaches destination without protected markers |

## Methodology

### Scenario Construction

OpenART establishes a 50-domain taxonomy from O\*NET occupational categories and interactive agent benchmarks. The planner constructs a directed workflow graph $G_q = (V_q, E_q)$ where vertices denote grounded environment operations and edges represent dependencies. The workflow satisfies:

$$
\begin{array}{c} (u, v) \in E _{q} \Rightarrow r _{q} (u) <   r _{q} (v), \\ I _{q} (v) \subseteq R _{q} \cup \bigcup_ {u \in \operatorname{Anc} _{G _{q}} (v)} O _{q} (u), \\ h (G _{q}) \in B (c), \end{array}\tag{1}
$$

where $r_q$ is a topological ordering, $R_q$ is the initial resource set, and $h(G_q)$ measures workflow complexity (size, dependency depth, parallel width).

The initial environment is compiled as:

$$
x _{0, q} = F (S _{q}, G _{q}, D _{q}), \qquad A (x _{0, q}; S _{q}, c) = 1.\tag{2}
$$

The evaluator is validated to correctly distinguish safe from unsafe outcomes:

$$
E _{q} (o _{q} ^{\mathrm{safe}}) = 0, \qquad E _{q} (o _{q} ^{\mathrm{unsafe}}) = 1.\tag{3}
$$

### Cross-Agent Runtime Projection

Each runtime $r$ is associated with an adapter $\psi_r$ that projects shared scenarios into native interfaces. During evolution, only authorized state changes are materialized:

$$
m _{t} = \Pi _{r} (\Delta _{t}) = \{\psi _{r} (a): a \in \Delta _{t},   v (a) \in V _{A} \cap V _{r}, p _{r} (a) \in L _{r} (v (a)),   \nu _{r} (a) = 1 \}.\tag{4}
$$

The attacker and target execute in **separate Docker containers**, making projection the only communication channel:

$$
(\xi _{t}, x _{t + 1}) \sim K _{r} (\cdot | \tau , x _{t}, m _{t}), \quad Y _{t} = E _{q} (\tau , \xi _{t}, x _{t + 1}).\tag{5}
$$

### Evolutionary Markov Hypergraph Attack (EMHA)

**Controlled Environment Evolution** follows the protocol:

$$
\begin{array}{c} {\Delta _{t} \sim p (\cdot | x _{t}, C _{t}),} \\ {m _{t} = \Pi _{r} (\Delta _{t}),} \\ {(\xi _{t}, x _{t + 1}) \sim K _{r} (\cdot | \tau , x _{t}, m _{t}),} \\ {Y _{t} = E _{q} (\tau , \xi _{t}, x _{t + 1}),} \\ {C _{t + 1} = U (C _{t}, x _{t}, \Delta _{t}, m _{t}, Y _{t}).} \end{array}\tag{6}
$$

Key properties:
- **Frozen parameters**: $\theta_{t+1} = \theta_t = \theta_0$ (Eq. 7)—no parameter updates
- **Hypergraph representation**: Vertices denote attack subgoals; hyperedges connect prerequisites to successors
- **Markovian path evolution**: Ready set depends only on current active subgoals, selected hyperedge, graph, and attacker state
- **Feedback-guided learning** using softmax policy over Q-values with temperature $\tau_Q$:

$$
V _{C} (q) = \tau _{Q} \log \sum_ {e \in R (q; G _{t})} \exp (Q _{C} (q, e) / \tau _{Q}), \quad \pi _{C} (e \mid q, x _{t}, G _{t}) = \frac {\exp (Q _{C} (q , e) / \tau _{Q})}{\sum_ {e ^ {\prime} \in R (q ; G _{t})} \exp (Q _{C} (q , e ^ {\prime}) / \tau _{Q})}.\tag{11}
$$

- **Credit redistribution** for delayed rewards:

$$
\delta _{t} = \max_{s \leq t} Y _{s} - \max_{s < t} Y _{s}, \quad \widetilde{r} _{t, j} = a _{t, j} \delta _{t}, \quad Q _{C} (q _{j}, e _{j}) \gets (1 - \alpha) Q _{C} (q _{j}, e _{j}) + \alpha [ \widetilde{r} _{t, j} + \gamma V _{C} (q _{j + 1}) ].\tag{13}
$$

- **Archive-guided graph evolution** following MAP-Elites quality-diversity search, with offspring generated via two graph-edit kernels.

## Empirical Validation / Results

### Benchmark Complexity Comparison

**Table 3: Task-level complexity comparison** (median [IQR])

| Benchmark | Tool calls | Dependency depth | Parallel width | State objects | File formats |
|-----------|-----------|-----------------|----------------|--------------|--------------|
| InjecAgent | 1 [1-1] | 1 [1-1] | 1 [1-1] | 1 [1-1] | 0 [0-0] |
| ToolEmu | 3 [1.5-4] | 2.5 [1.2-3.8] | 1 [1-1.8] | 3 [1.2-3] | 0 [0-0] |
| AgentDojo | 2 [1-3] | 2 [1-3] | 1 [1-1] | 1 [1-2] | 0 [0-0] |
| AgentHarm | 3.5 [3-4] | 3 [3-3] | 1.5 [1-2] | 3.5 [3-4] | 0 [0-0] |
| ASB | 2 [2-2] | 2 [2-2] | 1 [1-1] | 2 [2-2] | 0 [0-0] |
| DTap | 15 [7.4-18.7] | 2 [1-3] | 1.5 [1-2] | 2.5 [1-4] | 1 [0-3] |
| **OpenART** | **97 [90.2-100]** | **32 [15.8-84.8]** | **12.5 [3-24.5]** | **96.5 [90.2-100]** | **7.5 [7-9]** |

### Quality Validation

- **99.3% correctness** of evaluators under human expert verification (10% audit)
- All scenarios pass deterministic evaluator with automated behavioral probes

### Benign Task Completion

Pooled completion ranges from **80.81% (Qwen-3.7-Max)** to **96.18% (Claude Opus-4.8)**. Aider achieves the lowest completion rate (70.74% average), indicating a capability gap.

### Red-Teaming Effectiveness (Strict ASR)

**Table 6: Strict ASR across target agents and models**

| Target agent | GPT-5.5 | Opus-4.8 | GLM-5.2 | Qwen-3.7-Max | DS-V4-Pro | Avg. |
|-------------|---------|----------|---------|--------------|-----------|------|
| OpenCode | 100.0 | 62.5 | 91.2 | 98.9 | 99.1 | 90.3 |
| Aider | 61.2 | 38.2 | 62.3 | 66.6 | 66.7 | 59.1 |
| Claude Code | 93.6 | 64.6 | 92.3 | 99.5 | 98.7 | 89.7 |
| Codex | 85.8 | 58.4 | 84.2 | 90.3 | 91.4 | 82.0 |
| Continue CLI | 89.1 | 59.7 | 89.3 | 98.0 | 99.4 | 87.1 |
| Copilot CLI | 90.3 | 65.4 | 90.9 | 100.0 | 98.9 | 89.1 |
| CodeWhale | 92.0 | 61.8 | 90.8 | 99.0 | 99.2 | 88.6 |
| Goose | 83.7 | 54.8 | 82.4 | 89.0 | 88.9 | 79.8 |
| Hermes | 91.5 | 59.1 | 90.1 | 97.9 | 99.0 | 87.5 |
| Kilo | 89.6 | 65.4 | 89.7 | 98.9 | 97.8 | 88.2 |
| Nanobot | 91.3 | 61.7 | 92.1 | 97.1 | 98.5 | 88.1 |
| Oh My Pi | 94.3 | 59.3 | 93.6 | 99.8 | 99.3 | 89.3 |
| OpenClaw | 85.6 | 52.5 | 83.2 | 86.6 | 88.2 | 79.1 |
| Pi | 92.1 | 58.5 | 93.0 | 98.5 | 98.8 | 88.2 |
| Qwen Code | 90.9 | 65.2 | 92.5 | 98.7 | 99.5 | 89.4 |
| **Average** | **88.5** | **59.2** | **87.9** | **94.6** | **94.7** | **85.0** |

### Ablation Study Results

- **Workspace evolution**: 92.5% Strict ASR (10.9% above instruction-only)
- **Full EMHA**: 94.7% (13.1% above instruction-only, 2.2% above best single-vector)
- Removing archive: **−3.9%** ASR
- Removing credit redistribution: **−2.6%** ASR

### Key Insights

**Insight 1: Environment evolution exposes vulnerabilities missed by static evaluation.**
- Cumulative Strict ASR increases from 42.9% (round 1) to 94.7% (round 5)—a **51.8% cumulative improvement**
- Full EMHA's advantage over instruction-only grows from 1.8–2.7% to 17.2–17.6% as complexity increases

**Insight 2: Long-horizon execution amplifies environment changes.**
- Median propagation distance: **37 target actions** (IQR: 20–66)
- First consumption of evolved state at 23% of execution; first unsafe output at 64%
- Median latency of **41% of the workflow**

**Insight 3: Three recurring vulnerabilities:**
1. **Stale assumptions are rarely invalidated** (plan–referent drift, capability rebinding, checkpoint expiry)
2. **Safety decisions are propagated rather than recomputed** (deferred safety judgments to downstream components)
3. **Risk emerges compositionally over long workflows** (multiple benign changes interact to produce unsafe behavior)

## Theoretical and Practical Implications

### Theoretical Implications

- **Safety as a trajectory property**: The work provides strong empirical evidence that agent safety cannot be characterized through short, static, or model-centric evaluations. Safety must be understood as a property of long-horizon interactions between agents and their evolving environments.
- **Environment evolution as a first-class evaluation dimension**: By keeping task objectives and safety contracts fixed while evolving only environment state, OpenART enables causal attribution of safety failures to environmental factors.
- **Agent implementation matters beyond foundation model**: The finding that target-agent identity explains an additional 7.6% of ASR variation (beyond model and capability controls) suggests that runtime implementation—interfaces, state handling, memory management—plays a significant role in safety outcomes.

### Practical Implications

- **Benchmark design**: OpenART demonstrates that realistic agent safety evaluation requires long-horizon (median 97 tool calls), stateful scenarios with persistent state manipulation.
- **Attack surface coverage**: The 8 target-visible attack vectors (Workspace, Instructions, Skills, Tools, MCPs, Short-Term Memory, Plan State, Long-Term Memory) provide a comprehensive taxonomy for safety auditing.
- **Cross-agent comparability**: The target-agnostic scenario representation with lightweight adapters enables fair comparison across heterogeneous agent runtimes.
- **Adaptive red teaming**: EMHA provides a parameter-free, black-box reference policy that can be extended with arbitrary search strategies.

## Conclusion

OpenART establishes a scalable foundation for studying agent safety in complex, evolving environments. Key takeaways:

1. **Environment evolution is essential**: Static evaluation systematically underestimates agent risk, especially as scenario complexity grows (17.2–17.6% ASR gap on complex scenarios).
2. **Long-horizon workflows amplify safety failures**: Environment changes often remain latent for 41% of execution before manifesting as unsafe behavior.
3. **Recurring vulnerabilities exist across agents**: Stale assumptions, propagated safety decisions, and compositional risk are shared failure modes across diverse agent–model pairs.
4. **Future directions**: The work suggests the need for safety mechanisms that recompute rather than propagate safety decisions, invalidate stale assumptions, and monitor compositional risks over long workflows.

The OpenART framework is policy-agnostic, supporting arbitrary black-box search strategies beyond EMHA, making it a flexible foundation for continued research in agent safety evaluation.

---

_Markdown view of https://picx.dev/p/Ld0o52, served by PicX — AI-generated visual whiteboard summaries of research papers._