Architectural Convergence in Three LLM Agent Harnesses: A Source-Level Multiple-Case Study

Summary (Overview)

This paper presents a source-level, multi-case study of three open-source coding-agent harnesses—deepagents (LangChain), pi (Earendil Works), and dsh (DeepSeek)—selected for maximal philosophical spread (batteries-included, minimalist, everything-is-a-plugin).
The central finding is that despite opposing starting philosophies and opposite evolutionary trajectories (deepagents subtracts scaffolding; pi accretes infrastructure; dsh reuses pi's code), all three have converged on a single architectural "middle form" comprising five recurring elements.
The five convergent elements are: (1) a commoditised loop, (2) an append-only replayable session record, (3) model quirks externalised as data, (4) progressive disclosure of context, and (5) explicit extension seams.
The paper identifies one load-bearing dimension with no convergence and no presence: external verifiability—a tamper-evident record an outside party can check without trusting the runtime—which it frames as a predictive gap and the next axis of competition.
The study also documents four recurring "fault lines" (defect patterns) across the harnesses and decomposes the convergence mechanism into parallel discovery, diffusion, and literal reuse.

Introduction and Theoretical Foundation

Background

The paper positions the harness—the code surrounding a language model that builds context, mediates tools, runs the loop, and persists state—as the binding constraint on agent performance. Recent work formalises this: single-harness changes move Terminal-Bench 2 pass@1 by several points and SWE-bench Verified by up to fifteen points with the model held fixed.

Research Questions

RQ1 (Divergence): From what architectural positions did the harnesses start, and along what trajectories have they evolved?
RQ2 (Convergence): Is there a common form they are arriving at; what is it; and why are they converging?
RQ3 (Boundary): Is there a dimension on which they have not converged, and what explains the gap?

Theoretical Framing

The paper builds on prior work establishing that the agent-computer interface shapes capability (tool-centric agents), that scaffolding can determine performance more than model choice, and that multi-agent frameworks established the harness as reusable infrastructure. It contrasts this bottom-up empirical line with top-down prescriptive protocols (MCP, A2A, Autogenesis) that specify what harnesses should provide.

Methodological Lineage

The study follows established guidance for multiple-case study design in software engineering and treats source-level architectural narrative in the tradition of open-source architecture studies. It inherits methodological discipline (pinned commits, line-level evidence) from a prior taxonomy of thirteen coding-agent scaffolds.

Methodology

Study Design

Type: Explanatory, theory-building multiple-case study with literal replication logic across maximum-variation cases.
Case selection: deepagents and pi ground the candidate model; dsh serves as a held-out check (confirmatory only where its instantiation is independent, since dsh reuses pi's provider catalogue).
Pinned revisions: deepagents 0.7.8 @ commit 2c8015378; pi main @ a470b121b; dsh @ dsh-v0.1.1-rc.2, commit b150a551b.

Data Sources

Source code at pinned revisions, with claims anchored to file:line references.
Commit/PR/issue archaeology for trajectory evidence.
Hands-on reproduction: two plugins written against dsh's runtime; two defects (deepagents path-normalisation bypass, pi throttle-misclassification) independently re-derived in a sandbox.
Upstream confirmation: one documentation fix accepted and merged; one defect filed (issue #5640).

A Priori Dimensions

The comparison dimensions were fixed before close reading to avoid fitting a framework to observations. The study adds two axes lacking in prior work (auditability, evolution) and refines two (session-record strength, recovery semantics) from binary presence to graded form.

AI-Assisted Reading

The source sweep used parallel AI agents to locate and excerpt code; every claim was verified by hand. Reading depth was asymmetric: deepagents and pi were read line-by-line; dsh received one thorough pass plus a hands-on plugin experiment.

Divergent Origins (RQ1)

deepagents: The Subtracting Maximalist

Position: An assembly layer over LangChain's runtime; planning, filesystem, subagents, summarisation, memory, skills, permissions, and human-in-the-loop are all middleware composed in a three-phase stack (graph.py:816–893).
Architecture: Storage is a backend protocol with methods defaulting to NotImplementedError; context is defended in four automatic layers (delta-channel reducer, LLM summarisation near 85% of window, eviction of oversized tool results, truncation of stale tool arguments).
Trajectory: Subtracts authored scaffolding. The base prompt is deprecated ("no longer provides an authored base prompt"); PR #4859 stripped built-in tool-usage prose; PR #4929 removed todo/planning middleware from the default stack.

pi: The Accreting Minimalist

Position: Public identity is minimalism; the agent package is 2,368 lines, readable end-to-end, with no try/catch in the loop.
Architecture: A 2,941-line formal specification defines three durable forms (immutable entries, mutable registers, append-only usage rows) plus a durable "program counter" for crash recovery.
Trajectory: Accretes infrastructure. The repository is 146,170 lines across ten packages. The harness that rejected subagents now reserves "lanes" in its session model; the one that rejected compaction runs two implementations; the one that dismissed per-model coaching maintains a 2,998-line generator producing a detailed model-quirk catalogue.

dsh: The Plugin Absolutist

Position: TypeScript monorepo of ~230 plugins over a vendored service-locator runtime.
Architecture: A plugin is any object implementing a service; consumers resolve by stable context key; load order is declarative; registration is reversible. Even the agent loop is a configuration row (ctx.agentLoop).
Trajectory: Reuses. Its generic provider adapter (llm-pi-ai) depends directly on pi's published package, described in-tree as a "design-verification twin."

The Convergent Form (RQ2)

Five Recurring Elements

(1) A Commoditised Loop

pi's agent-loop file: 796 lines; dsh's driver: 515 lines; deepagents delegates to LangGraph.
The element is not loop size but that none competes on the loop—it is small, readable, or delegated wholesale.
Notable detail: pi's never-throw stream contract; provider failures become a terminal assistant message with stopReason of error or aborted.

(2) An Append-Only, Replayable Session Record

Strength ordering: deepagents (weak—compaction kept as a view, recovery rewrites messages) < pi (medium—appends CompactionEntry with firstKeptEntryId) < dsh (strong—event-sourced log with runtime-asserted invariant that anything a model saw must be reconstructable).

(3) Model Quirks as Data

One idea in three serialisations: class hierarchy (deepagents), data directory (pi), runtime resolver (dsh).

(4) Progressive Disclosure of Context

All three put only name, description, and path in the prompt; read body on demand. dsh goes furthest with written per-capability contracts stating token and KV-cache costs.

(5) Explicit Seams Instead of a Monolith

deepagents: three orthogonal axes (middleware, backend, profile); pi: flat event bus plus registries; dsh: named service-definition triples.
Disagreement is composition style, not whether seams should exist.

Table 2: Nine Dimensions Across Three Harnesses

Dimension	deepagents	pi	dsh
Philosophy	batteries-included middleware	minimal loop, extend outward	everything-is-a-plugin
Loop (commoditised)	none owned (delegates to LangGraph)	796-line loop file, no try/catch	515-line driver, itself a swappable config row
Append-only session	delta-channel; compaction kept as a view (weak)	append-only entries + registers (medium)	event-sourced + runtime "visible⇒logged" assertion + replace-op (strong)
Crash recovery	checkpoint; resume rewrites history	durable register; read-and-resume	closes an interrupted turn; refuses mid-stream corruption
Model quirks as data	profile class hierarchy	compat data catalogue (defaults inferred)	adapter config + runtime capability resolution
Progressive disclosure	skills index (name/desc/path)	CLI tools + READMEs, read on demand	projection + per-capability token/KV contract
Explicit seams	middleware / backend / profile (ordered)	event bus + registry (enumerated)	service-definition triples (locator)
Sandbox	permissions + HITL + paid remote backends	none ("YOLO"; containerise yourself)	bwrap/Landlock; throws rather than degrade
External verifiability	none (outsourced telemetry)	none (mutable private files)	operational only (not third-party checkable)

Three Findings

All five elements recur despite opposing philosophies—consistent with the middle form being a property of the problem.
No element recurs as the same code—convergence of idea, not implementation.
Only the session record forms a strength ladder rather than a cluster, pointing toward the non-converged dimension.

Residual Disagreement: Audience, Not Architecture

The remaining difference (how much to automate vs. leave observable) reflects audience: deepagents serves product-embedded users; pi serves a person at a terminal.

Mechanisms of Convergence

Parallel discovery: Same shape from different starting prose and pressures (e.g., append-only record from crash pressure vs. checkpoint-cost pressure).
Diffusion: Public and mutually known projects; shared conventions like AGENTS.md spread by imitation.
Literal reuse: dsh mounts pi's provider catalogue wholesale—the strongest, most concrete form.

Convergent Fault Lines

Four recurring defect seams were identified across deepagents and pi:

Seam	Specimen	Mechanism → Consequence	Loss
S1 sync/async drift	da, async_subagents	guard inside try in sync twin, outside in async twin → unhandled KeyError on recovery	○
S2 normalisation trust gap	da, utils.py:691	//secrets ≠ /secrets by path component but = by filesystem → checks fail open	●
S2 normalisation trust gap	pi, overflow.ts:75	throttle rethrown as bare object, JSON-stringified past caret-anchored regex → destructive compaction	●
S3 string-matched semantics	pi, overflow.ts:60	error meaning decided by ~25 regexes → reworded rate-limit misread as context overflow	●
S4 silent vs. loud failure	all four specimens	unrecognised shape defaults and runs on instead of stopping	—

Key rules derived:

S1: Derive one twin from the other or share a single guarded core.
S2: One mandatory canonicalisation point per trust boundary.
S3: Errors carry a typed kind; regex is a last-resort fallback that must fail toward the safe side.
S4: At every boundary, an unrecognised shape is an error, not a default.

The Boundary: A Dimension Without Convergence (RQ3)

The Verifiability Ladder

Baseline: No durable record.
deepagents: Checkpoints exist for resumption, not evidence; resume path rewrites history.
pi: Sessions append-only on newer path, but remain mutable files owned by the process.
dsh: Climbed highest—durable approval records, visible-implies-logged invariant—but aims at operational reconstructability, not external verifiability. Its own docs: ctx.tools.restrict() is "a visibility composition, not a permission boundary."
Empty rung: No tamper-evident record an outsider can check without trusting the runtime.
Empty rung: No first-class treatment of "what was this data as of," "may this result leave the building," or "prove the cost ceiling held."

Interpretation

The absence is read as a predictive gap, not an oversight. Protocol designs like Autogenesis make versioned lineage their centrepiece, so auditability is actively prescribed top-down while absent bottom-up—a selection-pressure asymmetry. The three converged on everything a developer at a terminal needs; verifiability is precisely what that audience does not need.

Discussion and Implications

For Harness Builders

The five elements are a checklist paid for three times under one selection pressure. Advice is scoped to the audience all three serve (a developer at a terminal in an attended, long-horizon session): adopt the settled shapes; do not defend your own version.

For the "Less Harness" Thesis

The lesson is not "thinner everywhere" but "thinner in coaching, thicker in durability"—deepagents subtracts coaching while pi accretes durability infrastructure.

For Evaluation

If the harness is the binding constraint and harnesses are converging in form, residual performance differences increasingly live in un-converged dimensions (recovery semantics, quirk coverage, verifiability).

Threats to Validity

External validity: N=3, all coding-agent harnesses read at one point in time (August 2026); mitigated by maximal-spread selection and pinned revisions.
Internal validity: The three are not fully independent (dsh depends on pi); handled by decomposing the mechanism rather than assuming independence.
Construct validity: Dimensions were fixed a priori; the five elements were induced during reading (marked as primary construct risk). The verifiability dimension aligns with the first author's research interest, disclosed for weighting.
Reliability: Single analyst with AI assistance; dsh read less deeply (claims marked as documentation-level where line evidence was not reached). A scaled companion study is underway with blind detectors and inter-run agreement measurements.

Conclusion

Three harnesses began from opposing philosophies and evolved in opposite directions, yet converged on one middle form. The convergence is real but not independent (parallel discovery, diffusion, literal reuse). When separate teams under the same pressure keep landing on the same shapes, those shapes are settled—the engineering move is to adopt them. The unsettled dimension is external verifiability, where all three are not merely divergent but absent. That is where the next harness will differ.