Summary of "ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement"

Summary (Overview)

• Proposes ModularRSI, a novel framework for recursive self-improvement (RSI) of agent harnesses that addresses the credit-assignment problem through contrastive trajectory analysis and modular decomposition.
• Introduces a benchmark-disjoint evolution protocol with 2,000 independently curated executable tasks, ensuring that harness improvements generalize beyond the evolution experience rather than adapting to benchmark-specific patterns.
• Decomposes the harness into five functional modules (Agent Loop, Tool Use, Observation Management, Context Management, Task Completion Detection) that are evolved independently and then integrated, reducing task-specific adaptation and cross-mechanism interference.
• Demonstrates consistent improvements on TerminalBench 2.0 and SWE-Bench Verified across unseen in-domain and cross-domain tasks, as well as transfer across different foundation models (DeepSeek-V4-Flash, GLM-5.2, MiniMax-2.5).
• Shows that modular evolution substantially outperforms both joint whole-harness evolution and non-modular evolution, with different modules contributing complementary improvements to execution reliability and interaction efficiency.

Introduction and Theoretical Foundation

Background and Motivation

CLI agents have achieved remarkable performance on complex software engineering and terminal-based tasks (Jimenez et al., 2024a; Deng et al., 2025; Merrill et al., 2026b). Beyond foundation models, their effectiveness increasingly depends on agent harnesses—the mechanisms that govern execution, tool interaction, context management, and environment feedback. Recent work has explored Recursive Self-Improvement (RSI) of harnesses, allowing agents to refine these mechanisms from execution experience.

Three Coupled Challenges

The paper identifies three fundamental challenges in achieving generalizable harness RSI:

1. Data-level challenge: Obtaining high-quality evolution experience is costly and difficult. Existing methods typically rely on data from downstream benchmarks, making it hard to distinguish reusable improvements from benchmark-specific adaptation.

2. Trajectory-level ambiguity: Individual execution trajectories entangle systematic harness deficiencies with task-specific reasoning and solution details, making it difficult to identify what should be improved.

3. Mechanism-level credit-assignment problem: Even when recurring behavioral deficiencies are identified, localizing them to responsible components within a monolithic harness remains difficult. Whole-harness optimization can entangle unrelated mechanisms.

Theoretical Foundation

The paper builds on the concept of contrastive learning—comparing successful and failed trajectories of the same task to isolate behavioral differences that are causally related to outcomes. This is combined with modular decomposition of the harness into functionally distinct components, enabling localized credit assignment and targeted evolution.

The framework translates coarse task-level outcomes into localized harness evolution signals by:

• Contrasting successful and failed trajectories for the same task
• Aggregating evidence across tasks to identify recurring behavioral deficiencies
• Evolving each module within a restricted modification scope

Methodology

Harness Modularization

ModularRSI decomposes the evolvable harness into five functional modules:

1. Agent Loop: Controls the iterative reasoning-action-observation process, including execution flow, interaction control, and recovery behavior.
2. Observation Management: Processes environmental feedback by preserving task-relevant information while filtering or compressing noisy observations.
3. Tool Use: Manages the selection, invocation, and validation of external tools.
4. Context Management: Maintains and organizes interaction history across execution steps.
5. Task Completion Detection: Determines whether the task has been completed or further interaction is required.

The framework uses Terminus-2 from Harbor (Merrill et al., 2026b) as the initial harness and reorganizes its behavioral mechanisms into these five modules.

Contrastive Trajectory Sampling and Analysis

For each evolution instance $x_i$, the agent is rolled out $K$ times, obtaining trajectories $(tra^1_i, tra^2_i, \ldots, tra^K_i)$. Each trajectory is assigned a binary reward $r^k_i$ where $r^k_i = 1$ indicates success. Tasks are divided into three groups:

$$G_i = \begin{cases} \text{Positive}, & \sum_k r^k_i = K, \\ \text{Contrastive}, & 0 < \sum_k r^k_i < K, \\ \text{Negative}, & \sum_k r^k_i = 0. \end{cases}$$

A Trajectory Memory stores historical trajectories and rewards across evolution epochs, providing additional contrastive evidence when current rollouts are insufficient.

Module-wise Harness Modification

Two mechanisms improve modification reliability:

• Modification Target Selection: Semantically similar diagnoses targeting the same function are consolidated into candidates, each assigned a vote count based on the number of distinct tasks providing supporting evidence. Higher-ranked candidates are prioritized.

• Evolution History: Maintains records of previous code changes and functionality introduced by each revision, reducing redundant or conflicting modifications and mitigating evolution oscillation.

Validation Gates

Only validated modifications are retained through three gates:

1. Program Check: Static checks including AST validation, import checks, protocol compliance, discovery-contract verification, and static self-attribute audits.
2. Diff Review: The Code-Modify Agent reviews each modification diff to check whether changes encode task-specific solutions unlikely to generalize.
3. Execution Validation: Two tasks are randomly sampled from the current batch and executed using the updated harness to detect runtime errors.

Cross-Module Integration

After the five modules are evolved independently, a cross-module integration epoch combines them into a unified harness, resolving conflicts by removing duplicated mechanisms, clarifying module responsibilities, and adjusting coordination logic.

Function Library Management

• Function Merge: Reduces redundancy by merging functions with highly similar or overlapping functionality.
• Task-Aware Function Composition: Dynamically constructs the active harness for each task by selecting task-relevant functions based on natural-language descriptions.

Empirical Validation / Results

Benchmark-Disjoint Evolution Dataset

The paper curates 2,000 executable evolution tasks from external data sources (GitHub, Hugging Face, Kaggle, Linux kernel documentation) that are fully disjoint from downstream evaluation benchmarks. The distribution spans 9 categories:

Generalization Beyond Evolution Experience

With DeepSeek-V4-Flash-Preview as the backbone model:

The harness evolved on TB-related data improves SWE-Bench Verified accuracy to 75.80, while the SWE-related harness improves TerminalBench 2.0 accuracy to 49.40, demonstrating cross-domain transfer.

Cross-Model Generalization

The evolved harness transfers across different foundation models on TerminalBench 2.0:

Effect of Modular Evolution

Single-module evolution results show all modules contribute improvements:

Comparison with Existing RSI Methods

ModularRSI improves accuracy by more than five points, while AHE and Meta-Harness remain close to baseline under the benchmark-disjoint protocol.

Effect of Contrastive Analysis

The proportion of contrastive trajectory pairs decreases across evolution epochs (from 36.67% to 34.17%), suggesting the harness progressively incorporates reusable behavioral improvements revealed by contrastive examples.

Effect of Evolution Data Quality

A Medium-centered difficulty distribution (50% of tasks in the 40-60% difficulty range) achieves 76.45% accuracy on SWE-Bench Verified, outperforming the Hard & Easy distribution (35% each in extreme ranges) by 2.20 percentage points (74.25%).

Theoretical and Practical Implications

Theoretical Contributions

1. Credit-Assignment Solution: ModularRSI addresses the fundamental credit-assignment problem in harness self-improvement by converting coarse task-level outcomes into localized function-level evolution signals through contrastive trajectory analysis and modular decomposition.

2. Benchmark-Disjoint Evaluation Protocol: Establishes a standardized evolution resource for studying transferable harness improvements, enabling evaluation of whether RSI methods learn generalizable improvements rather than adaptation to downstream benchmarks.

3. Modular Evolution Advantage: Demonstrates that independently evolving and merging harness modules substantially outperforms joint or non-modular evolution, providing evidence that restricting modification scope reduces interference when optimizing the harness.

Practical Implications

• Cross-Model Transfer: The evolved harness transfers across different foundation models, suggesting that improvements to agent execution mechanisms are model-agnostic and can be reused with different LLM backends.

• Execution Reliability: The pronounced improvement in Pass 3 (from 30.34 to 35.96 on TerminalBench 2.0) indicates that harness evolution enhances execution reliability, reducing stochastic failures across repeated trials.

• Efficiency Gains: Different modules provide complementary benefits—Observation Management substantially reduces execution steps (from 34.70 to 22.50 in single-module evolution), while Agent Loop yields the largest accuracy gain.

• Data Quality Insights: The difficulty distribution of evolution data plays a crucial role, with moderately difficult tasks providing more informative contrasts for discovering generalizable improvements.

Conclusion

ModularRSI presents a benchmark-disjoint, contrastive, and modular framework for generalizable harness self-improvement. By contrasting execution trajectories and evolving harness modules independently, it identifies reusable mechanism-level improvements while reducing task-specific adaptation. Experiments on TerminalBench 2.0 and SWE-Bench Verified demonstrate consistent gains across unseen tasks, domains, and foundation models.

Limitations

1. No dedicated ablation isolating the contribution of contrastive trajectory analysis (though supporting evidence is provided through trajectory analysis and case studies).
2. Due to computational cost, main evolution experiments use only a subset of the 2,000 curated evolution instances.

Future Directions

• Broader comparisons and larger-scale studies
• Further investigation of contrastive analysis contributions
• Exploration of the full 2,000-instance evolution dataset for more comprehensive evolution

The results demonstrate the potential of benchmark-disjoint, contrastive, and modular evolution for building more generalizable self-improving agent harnesses. All code and datasets are available at https://github.com/IQuestLab/ModularRSI.