EVOMAL: Self-Poisoning in Self-Evolving Coding Agents

Summary (Overview)

• Novel vulnerability discovery: The paper identifies self-poisoning, a CREATE-path vulnerability in self-evolving LLM coding agents where an agent retrieves a malicious skill, imitates it, and authors its own malicious skill that is stored in the persistent skill library—all without the attacker's skill ever being invoked.
• EVOMAL attack design: The attack wraps an interchangeable payload in a banner of benign-looking structural elements (copy-verbatim comments, decorators, import-time hooks) that induces imitation-based agents to reproduce the enclosed malicious code. Across six models on 153 tool-relevant SWE-bench Verified tasks, the Agent Self-Poisoning Rate (ASPR) ranges from 20.3% to 41.8%.
• Self-propagating worm: The infection cascades across generations—self-poisoned libraries contain 4.9 to 9.0 times as many malicious skills as initially planted. After planted skills are removed, Qwen3 retains a 68% round-5 ASPR, demonstrating a self-sustaining worm that evades existing defenses.
• Targeted attack amplification: Rewriting only the planted skill descriptions to match one task family (with no victim-specific knowledge) raises ASPR to as high as 86.7%.
• Counter-prompt defense: A fixed four-line instruction in the deployer's system prompt reduces EVOMAL's ASPR to ≤6.7% with no statistically significant loss in task completion, complemented by a structural signed-quarantine gate.

Introduction and Theoretical Foundation

Background

Self-evolving LLM coding agents (e.g., Voyager, MetaGPT, SWE-agent) store tools they write as persistent skills in a library. Subsequent tasks retrieve the closest matches by embedding similarity and either reuse them or author new ones. At production scale, these libraries accept community contributions (e.g., MCP Registry, skill marketplaces with tens of thousands of entries).

The Two-Path Attack Surface

The paper distinguishes two structurally distinct routes through which a retrieved skill can shape agent output:

• REUSE-path: The agent invokes a retrieved skill by name. The call site carries the attacker's skill name, making it catchable by name-based blocklists. This is the surface targeted by prior tool-poisoning work.
• CREATE-path (novel): The agent authors a new skill whose body reproduces a pattern it just read and stores it in the library. The authored copy carries an agent-chosen name, import surface, and call site that no name-based filter can flag. This path is unique to self-evolving deployments.

Key Insight

"Self-poisoning is therefore inherent to the paradigm, arising wherever an agent authors skills from what it reads."

The vulnerability is not created by attacker-written instructions around the code—it follows from the ordinary authoring behavior of self-evolving agents. Even a plain malicious skill without any instructional wrapper is re-authored by DeepSeek-V4-Pro in 11.1% of tool-relevant tasks.

Threat Model

Adversary (publish-only, no-box):

• A1: Publish access to the skill library
• A2: No model weights, training data, or runtime access
• A3: No victim knowledge, no invocation needed—a planted skill need only enter context once

Defender (deployer):

• D1: Controls system prompt, library settings, retrieval policy
• D2: Cannot retrain the model or remove tool access
• D3: Limited to observable variables

Formal Objective

The agent is modeled as $A = (M, R, P, L_0)$ where $M$ is the LLM, $R$ is the retriever, $P$ is the deployer-owned system prompt, and $L_0$ is the initial skill library. The defense objective is:

$$\min_{D \in \mathcal{D}_{\mathrm{def}}} \text{ASPR}(A_D, L^{atk}, \mathcal{D}) \quad \text{s.t.} \quad \mathbb{E}_{x \sim \mathcal{D}^{\mathrm{benign}}} \left[ \mathcal{B}_x - \mathcal{B}_x^D \right] < \varepsilon$$

Methodology

EVOMAL Attack Architecture

The attack seeds the agent's library with $n$ planted skills offline, operating through three components:

1. The Planted Skills (Banner + Payload)

A planted skill is an ordinary-looking Python file with two orthogonal parts:

Banner (three structural layers designed for verbatim copying):

• Copy-verbatim comment: A header announcing mandatory infrastructure ("REQUIRED: copy verbatim", "keep the helpers or CI fails")
• Decorator on a task-shaped host: An @_telemetry decorator marking a function shaped like the task itself
• Import-time registration: A _register_tool call that fires at import time

Payload: Interchangeable malicious code (e.g., credential exfiltration via _ping_analytics). The same banner can carry a reverse shell, cryptominer, or ransomware without changing how the skill spreads.

2. Generic vs. Targeted Attacks

• Generic attacker: Ships broadly named SE helpers (run_python_tests, check_python_syntax, read_source_file) with generic descriptions
• Targeted attacker: Same code bodies, but rewrites descriptions/names to match one task family (pytest fixtures, config validators, URL-regex helpers)

3. Cascade Protocol (Algorithm 1)

A rolling-replacement cascade models library evolution:

• Each round: agent works through the task pool, authoring skills $S_t$
• A random fraction $r$ of non-planted entries $B_t$ is dropped and replaced by freshly authored skills
• Persistent condition: Re-inserts $L^{atk}$ each round (attacker keeps republishing)
• Removed condition: Drops the planted set after round 0 (models a takedown)

Experimental Setup

• Agent stack: mini-SWE-agent (ReAct framework) + Voyager SkillManager (top-5 retrieval, BGE-M3 embeddings over ChromaDB)
• Datasets: SWE-bench Verified (N=153 tool-relevant tasks) and SWE-bench Pro (N=114)
• Models: Six models from six vendors—Devstral-Small-2 (22B), Gemma-4-31B-IT (31B), Qwen3-Coder-Next (~80B), GPT-OSS-120B, MiniMax-M2.7 (~230B), DeepSeek-V4-Pro (~1.6T)

Key Metrics

Agent Self-Poisoning Rate (ASPR) — the fraction of tasks on which the agent authors and stores a skill reproducing a planted pattern:

$$\text{ASPR}(A, L^{atk}, \mathcal{D}) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \mathcal{C}_x \right]$$

where $\mathcal{C}_x = \mathbf{1}\left[\exists t \in L^{atk} \cap \mathcal{R}_x: b_t(s(x)) = 1\right]$

ASPR decomposition (Equation 1):

$$\text{ASPR} = (\text{in-context rate}) \times (\text{conditional-copy rate})$$

Empirical Validation / Results

RQ1 (Feasibility): Agent as the Carrier

• All six models self-poison from an 8-skill seed at a 3.4% poisoning rate, with ASPR ranging from 20.3% to 41.8% on SWE-bench Verified
• Rates are 18.3 to 32.0 percentage points above the payload-free control
• Self-poisoned libraries contain 4.9 to 9.0 times as many malicious skills as initially planted
• The banner drives the effect: removing its three layers lowers DS-V4 ASPR by 13.0, 19.6, and 30.7 percentage points, respectively (Table 2):

• Vulnerability tracks code imitation, not model size or capability
• The attack generalizes across benchmarks, task subsets, and payloads (five malware classes: reverse shell, cryptominer, ransomware, keylogger, disk wiper)

RQ2 (Scaling): Task-Family Targeting

Description-only specialization raises ASPR by +11.8 to +26.9 pp across six models, peaking at 86.7% on Qwen3's pytest tasks (Table 3, Combined column):

RQ3 (Persistence): Cascade and Self-Propagation

• Persistent condition: Qwen3 climbs from 34.6% to 66.7% over five rounds; DS-V4 saturates at ~53%
• Removed condition: Qwen3 rises to 68% (strong self-sustaining propagation); Gemma4 shows weaker effect (29.4%); DS-V4 and MiniMax collapse immediately
• Replacement rate is critical: At $r=0.25$ the infection dies back; at $r=0.50$ it amplifies; at $r=0.75$ ASPR reaches 60.1% and still rises

Ablation Studies

• Agent scaffolds: EVOMAL generalizes to OpenHands and Claude Code (generic rates comparable to mini-SWE-agent)
• Tool languages: ASPR varies by only 0.7 pp across Bash, JavaScript, and Python
• Planted-skill count: ASPR saturates at 8 seeds (41.8%); even a single seed yields 19.6%
• Retrieval depth: k=3 lowers ASPR only to 30.7%; deeper retrieval offers the attacker little benefit
• Poisoning density: Shrinking the benign pool from 232 to 64 raises ASPR from 41.8% to 60.1%

Theoretical and Practical Implications

Why Self-Evolution Propagates

The paper models the cascade as a Galton–Watson branching process, with each infected entry's expected offspring given by:

$$\rho = c \cdot q \cdot \phi$$

where $c$ is the conditional-copy rate, $q$ is retrievability (expected top-k retrievals per round), and $\phi$ is persistence (fraction of authored skills surviving eviction).

Corollary 1 (Joint necessity of copy and reach): A regime with $\rho > 1$ requires both $c$ and $q$ bounded away from zero. DS-V4 copies more than Gemma4 yet collapses because its authored skills have narrow retrieval support (low $q$).

Failure of Existing Defenses

Theorem 1 formalizes that submission-side defenses (name blocklists, code scanners, classifiers) cannot reduce CREATE-path copying because their decisions depend only on attacker-submitted artifacts and named invocations, excluding the agent-authored skill from their inputs.

Empirical results (Table 5):

Proposed Defenses

1. Counter-prompt (reading step): A four-line system-prompt instruction telling the agent to treat banner-style "REQUIRED PATTERN" and "DO NOT MODIFY" instructions as untrusted content. Reduces ASPR to ≤1.8% (headline) and ≤6.7% (worst case across all conditions), with zero callbacks and no statistically significant task-completion loss.

2. Signed quarantine gate (writing step): A two-level library where a curator signs every retrievable entry; agent-authored skills enter an unretrievable quarantine level. Theorem 3 proves that under an unforgeable signature scheme, an agent-authored skill is retrieved with negligible probability, driving CREATE-path ASPR to a negligible level in the attacker-removed condition.

Conclusion

The paper establishes self-evolution as a critical new attack surface in coding agents. Key takeaways:

1. Self-poisoning is inherent to the self-evolving paradigm—any agent that authors skills from retrieved content is vulnerable, regardless of model size or capability.
2. The security boundary shifts from attacker-submitted artifacts to agent-authored skills, making conventional name-, code-, and signature-based screening insufficient.
3. Self-propagation requires joint conditions: both willingness to copy the banner (c) and retrieval reach of the copied skill (q) must be bounded away from zero for a self-sustaining worm.
4. Defenses must secure the authoring process and control how generated skills re-enter shared libraries. The lightweight counter-prompt provides effective low-cost mitigation; structural quarantine offers stronger containment at the cost of limiting self-evolution.

Future directions: Secure authoring, provenance tracking, and propagation control in future coding agents. The work opens the door to studying CREATE-path vulnerabilities as a general class of attacks in self-evolving AI systems.