# Harness Continual Learning: Continual Adaptation Beyond Model Parameters

> Harness Continual Learning enables frozen foundation models to accumulate capabilities by evolving prompts, memories, and tools around them, with guarded updates preventing harness-level forgetting.

- **Source:** [arXiv](https://arxiv.org/abs/2608.19013)
- **Published:** 2026-08-22
- **Permalink:** https://picx.dev/p/NkqbtS
- **Whiteboard:** https://picx.dev/p/NkqbtS/image

## Summary

## Summary

**Harness Continual Learning: Continual Adaptation Beyond Model Parameters**

**Authors:** Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao (Nanjing University; University of Wollongong)

---

## Summary (Overview)

- **New Paradigm:** The paper introduces **Harness Continual Learning (HCL)**, a novel continual learning paradigm where the "harness"—prompts, memories, tools, skills, and routing rules—evolves around a frozen foundation model, rather than updating model parameters.
- **Harness-Level Forgetting:** The authors formalize **harness-level forgetting**, a new type of catastrophic forgetting where updates to one harness component can disrupt previously reliable behavior in other components, even with a frozen model.
- **Guarded Evolution Framework:** They propose a "guarded harness evolution" mechanism with a Continual Optimizer that proposes candidate harnesses and a Continual Evaluator that commits changes only after verifying current improvement, historical retention, and validity.
- **Empirical Validation:** Across textual reasoning, multimodal perception, and open-world interaction tasks, HCL demonstrates capability accumulation and failure recovery, achieving relative gains exceeding 10% over baselines. The framework provides explicit control over the stability-plasticity trade-off.
- **Key Finding:** More permissive updates do not necessarily yield a stronger final harness; a moderate historical-loss tolerance (b=1) achieved the highest final average performance (63.46%) compared to both stricter (b=0: 61.25%) and looser (b=∞: 60.13%) settings.

---

## Introduction and Theoretical Foundation

### Background and Motivation

Traditional continual learning is **model-centric**, treating model parameters as the state that changes with sequential experience. However, modern AI agents adapt not only through model weights but also through an external **harness**—the persistent runtime contents that determine how a frozen foundation model receives information, retrieves experience, and acts.

> "Prompts, memories, tool and skill specifications, and routing policies can persist and evolve across interactions even when the foundation model remains frozen."

The authors identify a critical problem: harness components are **coupled in execution**. A memory update can change evidence retrieved for an earlier query; a skill revision can alter tool use; a routing edit can break a previously successful workflow. This creates **harness-level forgetting**—a degradation of earlier behavior caused by harness updates—which extends the classical stability-plasticity problem from model state to harness state.

### Key Definitions

The paper formalizes the HCL problem as follows. Let $H_n$ be the deployed harness at interaction step $n$, and $\theta$ the frozen foundation model parameters. The harness transforms raw interaction $\mathbf{u}_n$ into outcome $\mathbf{y}_n$:

$$\mathbf{e}_n = \left(\mathbf{u}_n, \mathbf{i}_n, \mathbf{z}_n, \mathbf{y}_n, \mathbf{f}_n\right)$$

where $\mathbf{i}_n$ is the structured interaction, $\mathbf{z}_n$ the execution context, and $\mathbf{f}_n$ post-execution feedback.

The Continual Optimizer proposes a candidate harness:

$$\widetilde{H}_{n+1} = \mathcal{O}_{F_\theta}\left(H_n, \mathbf{e}_n\right)$$

The deployed harness evolves based on commitment decision $G_n \in \{0,1\}$:

$$H_{n+1} = \begin{cases} \widetilde{H}_{n+1}, & G_n = 1, \\ H_n, & G_n = 0. \end{cases}$$

---

## Methodology

### Harness State Architecture

The HCL harness state is organized into four jointly versioned components:

$$H_n = \left(I_n, M_n, C_n, R_n\right)$$

**Table 1: Components of the HCL Harness**

| Component | Function during execution | Contents updated in HCL |
|---|---|---|
| Task Interface $I_n$ | Transforms raw interactions into structured representations | Prompts, task templates, and parsing and normalization rules |
| Experience Memory $M_n$ | Provides concrete interactions and abstract guidance for reuse | Raw interaction records and LLM-generated Abstract Memory entries |
| Capability Map $C_n$ | Provides external operations and reusable inner skills | Inner skills extracted from Abstract Memory |
| Adaptive Router $R_n$ | Selects and organizes memory and capabilities | Routing prompts, selection criteria, and workflow templates |

#### 1. Task Interface
Transforms raw input into structured representation:

$$\mathbf{i}_n = I_n(\mathbf{u}_n) = (\mathbf{x}_n, \mathbf{g}_n, \mathbf{k}_n)$$

where $\mathbf{x}_n$ is the available input, $\mathbf{g}_n$ the task objective, and $\mathbf{k}_n$ records constraints.

#### 2. Experience Memory
Dual-form memory combining:
- **Raw Memory** $M_n^{\text{raw}}$: concrete interactions (input, output, feedback)
- **Abstract Memory** $M_n^{\text{abs}}$: LLM-summarized guidance and recurring patterns

$$M_n = \left(M_n^{\text{raw}}, M_n^{\text{abs}}\right)$$

#### 3. Capability Map
$$C_n = \left(C_n^{\text{outer}}, C_n^{\text{inner}}\right)$$

- **Outer capabilities**: external APIs, retrieval services, environment actions
- **Inner capabilities**: reusable skills abstracted from Abstract Memory

#### 4. Adaptive Router
Connects all components to execution:

$$\mathbf{z}_n = R_n\left(\mathbf{i}_n, M_n, C_n\right)$$

### Guarded Harness Evolution

The framework separates update generation from state commitment through a **proposal–evaluation–commitment** process.

#### Continual Optimizer
Generates candidate harnesses from post-execution feedback, proposing up to $K$ alternatives per component in a sequential strategy.

#### Continual Evaluator
Three checks determine commitment:

**Current Improvement:**
$$\Delta_n = P\left(\widetilde{H}_{n+1}, V_n\right) - P\left(H_n, V_n\right)$$
Accepted when $\Delta_n \geq \delta_n$.

**Historical Retention:** Counts previously solved anchors that fail under candidate:
$$D_n = \sum_{a \in A_n} \mathbb{1}\left[q(H_n, a) = 1 \wedge q(\widetilde{H}_{n+1}, a) = 0\right]$$
Accepted when $D_n \leq B_n$.

**Validity Check:** $v_{n,\ell}\left(\widetilde{H}_{n+1}\right) \in \{0,1\}$ for each check $\ell$.

**Commitment Decision:**
$$G_n^{(k)} = \mathbb{1}\left[(\Delta_n^{(k)} \geq \delta_n) \wedge (D_n^{(k)} \leq B_n) \wedge \left(\forall \ell, v_{n,\ell}(\widetilde{H}_{n+1}^{(k)}) = 1\right)\right]$$

Two configurations are studied: **Stability-HCL** ($B_n = 0$, strict retention) and **Plasticity-HCL** ($B_n = \infty$, permissive updates).

---

## Empirical Validation / Results

### Experimental Setup

| Setting | Stream | Frozen Model |
|---|---|---|
| ALFWorld | 6 task categories | Qwen3.5-9B |
| Minecraft | 50-task curriculum | Qwen3.6-27B |
| Textual Reasoning | MuSiQue → ProofWriter → GSM8K → HotpotQA | DeepSeek-V4-Flash |
| Multimodal | Detection → Caption → Grounding → VQAv2 | Qwen3.6-27B |

### Open-World Interaction

**ALFWorld Results:**

| Method | Pick | Look | Clean | Heat | Cool | Two-object | Final Avg. ↑ | Avg. Fgt. ↓ |
|---|---|---|---|---|---|---|---|---|
| Static Harness | 95.80 | 66.70 | 25.80 | 26.10 | 9.50 | 58.80 | 47.12 | - |
| RAG Baseline | 95.80 | 83.30 | 41.90 | 39.10 | 14.30 | 58.80 | 55.56 | 1.74 |
| MemP | 95.80 | 83.30 | 48.40 | 34.80 | 9.50 | 47.10 | 53.15 | 5.18 |
| MemRL | 87.50 | 66.70 | 29.00 | 60.90 | 23.80 | 41.20 | 51.51 | 5.64 |
| **Stability-HCL (Ours)** | **100.00** | **83.30** | **51.60** | 30.40 | **28.60** | 76.50 | **61.74** | **2.64** |
| **Plasticity-HCL (Ours)** | **100.00** | 77.80 | 41.90 | **39.10** | 19.00 | **100.00** | **62.98** | 10.94 |

**Minecraft:** HCL completed all 50 tasks while the Static Harness plateaued at 15. HCL used only 83 environment actions versus 88 (MemRL) and 91 (MemP), demonstrating superior execution efficiency.

### Controlled Streams

**Textual Reasoning:**

| Method | MuSiQue | ProofWriter | GSM8K | HotpotQA | Final Avg. ↑ | Avg. Fgt. ↓ |
|---|---|---|---|---|---|---|
| Zero-shot | 35.00 | 42.80 | 49.40 | 54.80 | 45.50 | - |
| Stability-HCL | 27.60 | 73.00 | 50.40 | 57.80 | 52.20 | 0.00 |
| Plasticity-HCL | 29.00 | 77.00 | 92.00 | 60.80 | **64.70** | 0.07 |

**Multimodal Perception:**

| Method | Detection | Caption | Grounding | VQAv2 | Final Avg. ↑ | Avg. Fgt. ↓ |
|---|---|---|---|---|---|---|
| Zero-shot | 4.27 | 25.47 | 43.00 | 84.87 | 39.40 | - |
| DGG | 29.58 | 29.77 | 48.96 | 62.60 | 42.73 | 0.26 |
| Plasticity-HCL | 64.14 | 37.31 | 90.60 | 79.80 | 67.96 | 0.81 |
| **Stability-HCL** | **65.34** | **39.41** | **91.60** | 79.33 | **68.92** | **0.22** |

### Stability-Plasticity Trade-off

Varying only the historical-loss tolerance $B_n$:

| Tolerance | MuSiQue | ProofWriter | GSM8K | HotpotQA | Final Avg. ↑ | Avg. Fgt. ↓ |
|---|---|---|---|---|---|---|
| b = 0 | 27.83 | 73.33 | 84.33 | 59.50 | 61.25 | 0.39 |
| **b = 1** | 24.83 | 77.50 | 92.33 | 59.17 | **63.46** | 1.22 |
| b = 3 | 26.83 | 79.83 | 83.00 | 58.50 | 62.04 | 2.00 |
| b = ∞ | 28.33 | 71.00 | 82.00 | 59.17 | 60.13 | 3.45 |

### Component Ablation

| Method | Final Avg. ↑ | Avg. Fgt. ↓ |
|---|---|---|
| Zero-shot | 34.84 | - |
| w/o Interface update | 62.37 | 0.11 |
| w/o Memory update | 62.28 | 0.83 |
| w/o Capability update | 63.12 | 0.06 |
| w/o Router update | 62.77 | 0.14 |
| **Full HCL** | **63.41** | 0.45 |

---

## Theoretical and Practical Implications

### Theoretical Contributions

1. **New Learning Paradigm:** HCL reframes continual learning as operating on harness state rather than model parameters, extending the field's scope to agentic AI systems where the model is frozen but the surrounding infrastructure evolves.

2. **Formalization of Harness-Level Forgetting:** The paper provides a rigorous definition and measurement framework for harness-level forgetting, establishing it as a distinct phenomenon from classical catastrophic forgetting.

3. **Unified Framework:** HCL conceptually unifies multiple model-centric continual learning principles (replay, representation learning, architecture-based, optimization-based) within a single system-level formulation.

### Practical Implications

1. **Efficient Adaptation:** HCL enables capability acquisition without expensive model fine-tuning, making it practical for large-scale deployment with frozen foundation models.

2. **Controlled Trade-offs:** The explicit $B_n$ parameter gives practitioners direct control over the stability-plasticity trade-off, allowing task-specific tuning.

3. **Failure Recovery:** The framework supports recovery from failures through its memory and capability revision mechanisms.

4. **Scalability:** Demonstrated across model families (Qwen, DeepSeek) and scales (4B-27B), suggesting generalizability.

---

## Conclusion

**Key Takeaway:** Harness Continual Learning represents a paradigm shift in continual learning—moving the learning object from model parameters to the harness state surrounding frozen foundation models.

**Main Findings:**
- Harness evolution can accumulate capabilities and recover from failures without updating model parameters
- Harness-level forgetting is measurable and controllable through explicit retention constraints
- The stability-plasticity trade-off can be adjusted explicitly via the historical-loss tolerance parameter $B_n$
- More permissive updates do not necessarily produce stronger final harnesses

**Future Directions:**
- Efficient retention evaluation methods
- Harness-content consolidation techniques
- Evaluation over longer interaction streams
- Application to production agentic AI systems

The authors hope HCL "provides a foundation for addressing these challenges and encourages broader research on reliable agent continual learning."

---

_Markdown view of https://picx.dev/p/NkqbtS, served by PicX — AI-generated visual whiteboard summaries of research papers._