# Built on Broken Assumptions (BOBA): Why Coding Agents Cannot Be Left Alone

> Coding agents complete 95% of individual requests but only 51% of chains built on their own work, because unverified silent misses compound into broken foundations.

- **Source:** [arXiv](https://arxiv.org/abs/2611.02207)
- **Published:** 2026-11-07
- **Permalink:** https://picx.dev/p/VZgYzU
- **Whiteboard:** https://picx.dev/p/VZgYzU/image

## Summary

# Summary of "Built on Broken Assumptions (BOBA): Why Coding Agents Cannot Be Left Alone"

## Summary (Overview)

- **Core finding**: Coding agents complete 95% of individual requests but only 51% of chains of 20 consecutive requests built on their own prior work, revealing a "Built on Broken Assumptions" phenomenon where unverified mistakes compound.
- **Key contribution**: Introduces **BOBA-Bench**, a benchmark of 280 chains of 20 consecutive requests derived from 140 real open-source Python repositories (5,600 total requests), designed to evaluate agents in continuous, unattended development scenarios.
- **Root cause identified**: The degradation is not caused by worse code quality (verified agent code is nearly as good a base as human code) but by **silent misses**—mistakes agents don't notice because their own tests pass, which then become broken foundations for subsequent requests.
- **Verification gap**: Agents cannot self-verify—when asked to review implementations, they are correct only 53% of the time (barely better than a coin flip). A single external "verification failed" message plus one repair round lifts chain completion from 51% to 59%.
- **Implications**: Current benchmarks that test agents one request at a time on human-written code dramatically overestimate real-world performance in autonomous, multi-request development scenarios.

## Introduction and Theoretical Foundation

### Background and Motivation

Coding agents (e.g., Codex, Claude Code, OpenHands) are increasingly deployed as autonomous developers that handle chains of requests, where each request builds on the previous one's output. While trajectory studies confirm that agents:
- Rarely catch mistakes that nothing flags for them
- Report wrong work as complete

Existing benchmarks evaluate agents on:
- A single request
- Human-written code
- Verified once

This creates a fundamental mismatch between how agents are evaluated and how they're actually used in practice.

### Prior Work Limitations

The paper identifies gaps in existing repository-evolution benchmarks:

| Benchmark | Gap |
|------------|-----|
| SWE-Milestone, ChainSWE | Carry verified and unverified work forward together, conflating authorship and verification effects |
| StaminaBench, EvoCodeBench | Stop at first failure, never evaluating from unverified work |

**Key insight**: Unattended development can only be measured by *controlling what it lacks*: verification. The paper argues that the central variable should be whether code was verified, not who wrote it.

## Methodology

### BOBA-Bench Construction

The benchmark transforms real repository commit histories into chains of 20 requests through a four-stage process:

1. **Mine history**: Consecutive commits that change tests and source together become a request if new tests fail before and pass after the human patch
2. **Produce spec**: Claude Opus 4.8 writes specifications from test diffs, commit messages, and issues (never from source code)
3. **Audit specifications**: Three model families (GLM-5.2, Fable 5, GPT-5.6) must unanimously accept identical text
4. **Certify chains**: 20 consecutive requests form a chain, sealed with frozen dependencies, where human patches must pass every hidden test at every state

### Evaluation Protocol

The autonomous evaluation protocol delivers requests one at a time:
- Each request in a fresh conversation (agent remembers nothing of earlier requests)
- Graded against hidden tests after every request
- Agent told nothing about its performance
- All 20 requests run regardless of failures

### Evaluation Settings

Five experimental settings isolate different variables:

| Setting | Purpose |
|---------|---------|
| **Autonomous** | Full benchmark: agent left alone, no feedback |
| **Agent Code / Human Code** | Isolate authorship: verified agent code vs. human code as foundation |
| **Silent Continuation** | Isolate verification: issue next request on silent miss as if it passed |
| **Verified** | External "verification failed" message + one repair round after misses |
| **Review** | Model judges finished implementations without knowing author |

### Models Evaluated

Ten models on all 280 chains (8 attempts each): Claude Opus 5.5, Opus 5, Opus 4.8, Fable 5, GPT-6 Astra, GPT-6.1-sol, GPT-5.6-sol, GLM-5.3, GLM-5.3-Flash, Kimi K3.

## Empirical Validation / Results

### Chain Completion Findings

**Table 2: Chain completion of ten coding agents on BOBA-Bench**

| Model | pass@1 (%) | Single attempt (%) | Strict pass@1 (%) | Regression rate (%) | pass@8 (%) | Request completion (%) |
|--------|-----------|-------------------|-------------------|---------------------|------------|----------------------|
| Claude Opus 5.5 | 66.7 | 66.8 | 55.8 | 22.9 | 83.9 | 84.0 |
| Claude Opus 5 | 61.6 | 60.7 | 50.5 | 28.1 | 82.9 | 80.8 |
| GPT-6 Astra | 53.2 | 51.1 | 38.7 | 35.1 | 63.9 | 75.2 |
| Claude Fable 5 | 51.5 | 51.8 | 38.2 | 40.5 | 72.1 | 73.2 |
| GLM-5.3 | 51.2 | 46.4 | 39.8 | 30.1 | 76.1 | 73.0 |
| GLM-5.3-Flash | 51.1 | 51.1 | 40.9 | 30.1 | 77.5 | 73.9 |
| GPT-6.1-sol | 50.3 | 48.9 | 37.6 | 37.8 | 60.7 | 74.2 |
| GPT-5.6-sol | 45.2 | 45.7 | 33.3 | 42.2 | 60.0 | 71.8 |
| Kimi K3 | 44.2 | 45.4 | 33.7 | 42.8 | 68.2 | 68.8 |
| Claude Opus 4.8 | 39.2 | 38.2 | 27.5 | 42.4 | 60.0 | 66.2 |

**Key results:**
- Average chain completion: **51%** (vs. 66–84% individual request completion)
- Strict pass@1 (no request missed, no test broken): only **39%** on average
- Silent misses occur even within the first five requests across all models
- 23–43% of attempts break a previously passing test later in the chain

### Authorship vs. Verification Analysis

**Figure 4: Verified foundations hold whoever built them**

| Comparison | Result |
|------------|--------|
| Agent code vs. human code (both verified) | Gap: −3.6 to +0.8 percentage points (avg. −1 pp) |
| Unverified state vs. verified state | Failure rate nearly **triples** (avg. +18 pp) |
| Odds of failure per additional unverified mistake | Roughly **doubles** with each additional mistake |

**Key insight**: When code is *verified*, agent-written code is nearly as buildable as human code. The degradation comes from *unverified* mistakes, not from authorship.

### Failure Root Causes

Four primary causes of silent misses identified:

1. **Regressions** (most common): Agents break behavior the request didn't touch (37–55% of first failures)
2. **Missed requirements**: Part of specification left unimplemented (14% of failures)
3. **Wrong logic / integration errors**: Wrong implementation or unreachable code (18% of failures)
4. **Trusting own tests**: A green local test run barely lowers miss probability (8.3% vs. 8.6% for all submissions)

### Cross-Model Review Experiment

**Figure 5: No model is a reliable reviewer**

| Reviewer type | Wrong code accepted | Correct code rejected | Youden J |
|---------------|---------------------|----------------------|----------|
| Lenient (Claude models, GLM-5.3, Kimi K3) | 77–95% | 1–8% | ≤ 0.15 |
| Strict (GPT-6 Astra, GPT-6.1, GPT-5.6) | 40–55% | 41–58% | ≤ 0.04 |

**Key finding**: No model separates wrong from correct implementations well enough to serve as a check. The best Youden J is only 0.15 (where 1.0 is perfect).

### Effect of External Verification

**Table 41: Verified under the handoff protocol**

| Model | pass@1 Autonomous → Verified | Gain (pp) |
|--------|------------------------------|-----------|
| Claude Opus 5.5 | 66.8 → 73.9 | +7.1 |
| Claude Opus 5 | 60.7 → 69.6 | +8.9 |
| GPT-6 Astra | 51.1 → 53.9 | +2.9 |
| Claude Fable 5 | 51.8 → 63.9 | +12.1 |
| GLM-5.3 | 46.4 → 68.6 | **+22.1** |
| GLM-5.3-Flash | 51.1 → 67.1 | +16.1 |
| GPT-6.1 Sol | 48.9 → 52.9 | +3.9 |
| GPT-5.6 Sol | 45.7 → 46.8 | +1.1 |
| Kimi K3 | 45.4 → 52.5 | +7.1 |
| Claude Opus 4.8 | 38.2 → 44.6 | +6.4 |

**Key findings:**
- A single "verification failed" message + one repair round lifts chain completion by 1–22 points
- Naming the failing test achieves in one round what insisting achieves in ~3 rounds
- Repaired states are no longer broken foundations: next request fails on them no more often than on certified states
- False alarms (rejecting correct work) cost little: ~98% of phantom repair rounds leave work passing

## Theoretical and Practical Implications

### For System Builders

- **Keep the verification line open in both directions**: The verifier must see results; the model must receive verdicts
- **Widen what the verifier sees** rather than multiplying rounds: naming failing tests beats repeated generic rejection
- **Multi-agent systems are especially exposed**: models reviewing finished implementations barely beat chance, so the verifier's line must reach subagents directly

### For Model Builders

The paper calls for prioritizing **certification** as much as implementation:
1. Agents must tell wrong implementations from correct ones (currently barely better than chance)
2. Agents must repair a miss in one round when told only "verification failed" (currently repair only a third to a half)
3. Both must be achievable without a supervisor

### For Benchmark Builders

Three properties enable observing the phenomenon:
1. Agent's work persists across requests
2. Correctness decided by a contract the agent cannot see or rewrite
3. Requests depend on one another

Single-request benchmarks have none of these properties, which is why the phenomenon has gone unmeasured.

### For Users

Two practical recommendations:
1. **Check before you build on it**: Run or test a request's output before issuing the next request
2. **Say what you saw**: Even imperfect verification feedback helps; occasional false alarms cost little

## Conclusion

The paper demonstrates that coding agents build on broken assumptions: they complete most individual requests but only about half of the chains those requests form, primarily because they cannot detect their own mistakes. The problem is not the quality of code they write (verified agent code is nearly as good as human code) but their inability to verify their own work.

**Future directions**:
- Model builders should make certifying a request as much a goal as implementing it
- A single external "that's wrong" recovers part of the loss, but more than half of misses survive a repair round
- The benchmark and all artifacts are released at https://github.com/amazon-agi/BOBA-bench for reproduction and extension

**Bottom line**: Coding agents are only as reliable as the checks they can reach—without verification, every request that follows is built on broken assumptions.

---

_Markdown view of https://picx.dev/p/VZgYzU, served by PicX — AI-generated visual whiteboard summaries of research papers._