# CoReMem: Cost-Internalizing Memory for Successive Coding Tasks with Prospective Valuation

> CoReMem improves coding agent resolve per token by 41% on average by storing only repository anchors and training actors to use them, unlike existing memory methods that increase costs.

- **Source:** [arXiv](https://arxiv.org/abs/2611.17316)
- **Published:** 2026-11-14
- **Permalink:** https://picx.dev/p/iqKt76
- **Whiteboard:** https://picx.dev/p/iqKt76/image

## Summary

## Summary (Overview)

- **CoReMem** is a cost-internalizing memory framework for successive coding tasks on evolving repositories, addressing two key challenges: (1) determining what information is worth carrying across task boundaries before the future task is known, and (2) adapting the actor model to use memory efficiently during execution.
- The framework introduces **prospective segment valuation** to systematically value each part of a trajectory by what it saves on the next task, leading to a memory design built around **anchors** (repository locations and executable checks).
- **On-policy adaptation with provenance-gated value credit** trains the actor model to act on memory where it saves effort, using a reward that prioritizes resolution and penalizes step count only in successful runs.
- On ChainSWE, CoReMem raises the 27B actor's resolve rate from 21.0% to 25.7%, with 17% fewer steps and 11% fewer total cache-aware tokens relative to the untrained actor without memory.
- Across three open-weight actor models, CoReMem improves resolve per token over no memory by **41% on average**, while existing memory methods (Mem0, MemOp, ReasoningBank) fail to reach the quadrant where both steps and total tokens decrease.

## Introduction and Theoretical Foundation

### Background and Motivation

Real-world software development requires coding agents to handle successive tasks on an evolving repository, where each task inherits the repository state left by earlier tasks, and future tasks are unknown in advance. While prior research has studied repository issues independently, recent work considers sequences of tasks on shared, evolving repositories. Since repository state carries over, suitable memory can let agents reuse information acquired during earlier exploration and avoid rediscovering it on unseen future tasks.

### Two Key Challenges

**Challenge 1: What to carry across task boundaries.** Before the future task is known, what information is worth carrying? Existing memory methods are not designed for the successive-task setting:
- Within-task methods compact context for the current task whose goal is already known
- Cross-task methods distill general experience across independent tasks and repositories
- MemOp accumulates memory across tasks but resets repository state between tasks, optimizing memory based on gains on the source task rather than value to unseen future tasks

**Challenge 2: How to use memory during execution.** Memory's value is realized only when the actor model uses it appropriately. Existing methods optimize storage/retrieval but leave the reading model unchanged. An agent's information needs evolve as a task unfolds, making task-start retrieval insufficient.

### Learning Objective

The actor is trained from parameters $\theta_0$ to $\theta = \theta_0 + \Delta\theta$. The next task's session is $\tau \sim \pi_{\boldsymbol{\theta}}(\cdot \mid S_k, T_{k+1}, m)$. The optimization objective is:

$$
\begin{array}{r l} & {\underset {\Delta \theta}{\min} \mathbb {E} _ {\theta} [ n (\tau) ]} \\ & {\quad \mathrm{s.t.} \mathbb {E} _ {\theta} [ \mathrm{res} (\tau) ] \geq \mathbb {E} _ {\theta_ {0}} [ \mathrm{res} (\tau) ],} \end{array}\tag{1}
$$

where $n(\tau)$ is the step count, $\mathrm{res}(\tau) \in \{0, 1\}$ indicates whether the task resolves, and the expectation is over boundaries $b_k \in B$ and trajectories $\tau \sim \pi_\theta$.

## Methodology

### 3.1 Prospective Segment Valuation of Memory

**Prospective value of memory.** The value of memory $m$ at boundary $b_k$ is the change it causes on the next task's outcome $y$:

$$
\Delta_{y}(m \mid b_{k}) = \mathbb{E}[y(m)] - \mathbb{E}[y(\emptyset)],\tag{2}
$$

where $m = \emptyset$ means running with no memory. This is estimated as $\widehat{\Delta}_y(m \mid b_k) = \bar{y}(m) - \bar{y}(\emptyset)$ from mean outcomes, with $n$ candidates sharing one no-memory arm ($n+1$ arms instead of $2n$).

**Segment-by-segment valuation.** The trajectory is split into seven segments (reasoning text, commands, repository content read, test output, diff, outcome, scratch files). For each segment $c$, the value of removing it is computed:

$$
\begin{array}{r l} & {\nu_{c} = \mathrm{med}_{b} \left[ \widehat{\Delta}_{\mathrm{s}}(M \backslash M_{c}) - \widehat{\Delta}_{\mathrm{s}}(M) \right],} \\ & {\kappa_{c} = \mathrm{med}_{b} \left[ \widehat{\Delta}_{\mathrm{t}}(M \backslash M_{c}) - \widehat{\Delta}_{\mathrm{t}}(M) \right].} \end{array}\tag{3}
$$

where $\nu_c$ and $\kappa_c$ are the median changes in steps and tokens when segment $c$ is removed from the full trajectory $M$ used as memory.

**Value-guided memory design.** Removing commands or repository content adds steps on the next task, while removing reasoning text or diff body adds no steps and saves tokens. This leads to:

**Definition 1 (Anchor).** An anchor is a reference, taken from a trajectory, to a specific place in the repository (a location, e.g., `rules/ast_if.py:34-58`) or to a specific command that runs or tests the code (a check, e.g., `python -m pytest tests/ -k sim102`), which the next task can act on as it stands. $A(x)$ denotes the anchors that text $x$ contains.

The memory template keeps commands and repository content as main fields, test output/final diff/outcome in reduced form, and drops reasoning text and scratch files. The memory keeps the whole median step saving of the full trajectory with only **6% of its bytes**.

### 3.2 On-Policy Adaptation for Memory Use with Provenance-Gated Value Credit

#### 3.2.1 Cost-Internalizing On-Policy Valuation

**Cost-internalizing reward.** At a training boundary $b = b_k$, the policy $\pi$ rolls out a group of $G$ sessions from state $S_b = S_k$, with and without memory. Each session is scored:

$$
R_{i} = \left\{ \begin{array}{l l} W - n_{i} & \text{if session } i \text{ resolves } T_{k+1}, \\ \alpha f_{i} & \text{otherwise}, \end{array} \right.
$$

$$
W > n_{\mathrm{max}} + \alpha,\tag{4}
$$

where $n_i$ is the step count, $n_{\mathrm{max}}$ the step limit, $f_i \in [0,1]$ the fraction of fail-to-pass tests that pass after the session, $\alpha$ the weight of partial progress, and $W$ a payoff exceeding any step count plus $\alpha$. Every resolved session outranks every unresolved one, and effort is charged only inside success.

**Paired on-policy valuation.** The group mixes three kinds of sessions: memory-free (∅), own-memory (m), and one misplaced-memory session (m̃) carrying a memory from another repository. Each session receives group-relative advantage against memory-free sessions:

$$
A_{i} \stackrel{\varnothing, m, \tilde{m}}{=} \frac{R_{i} - \bar{R}_{\varnothing}}{s} \stackrel{m}{=} \frac{R_{i} - \bar{R}_{m}}{s} + \frac{\widehat{V}_{b}}{s},\tag{5}
$$

where $\bar{R}_{\emptyset}$ and $\bar{R}_m$ are mean rewards of ∅ and m sessions, $s$ is a fixed scale, and $\widehat{V}_b = \bar{R}_m - \bar{R}_{\emptyset}$ estimates the memory's on-policy value $\mathbb{E}_{m \sim q_b}[\Delta_R(m \mid b)]$.

#### 3.2.2 Step-Level Provenance-Gated Value Credit

**Provenance gate.** A step $t$ of an own-memory session is memory-driven ($\omega_t = 1$) when its message $x_t$ contains an anchor from the memory that was not seen before and is not found without memory:

$$
\begin{array}{c} \omega_{t} = \mathbf{1}\big[\exists a \in \mathcal{A}(x_{t}) \colon a \in \mathcal{A}(m), \\ a \notin \mathcal{A}(h_{t}), a \notin \mathcal{U}_{b}\big], \end{array}\tag{6}
$$

where $h_t$ is the task history before step $t$ (memory excluded) and $\mathcal{U}_b$ are anchors that memory-free sessions reach within their first few steps.

**Step-level value credit.** Each step $t$ of an own-memory session $i$ is credited:

$$
A_{i, t} = \frac{R_{i} - \bar{R}_{m}}{s} + \omega_{t}\frac{\widehat{V}_{b}}{s},\tag{7}
$$

Setting every $\omega_t = 1$ recovers the paired whole-trajectory credit of Eq. (5).

**Proposition 1 (Value credit as a push on memory use).** Under the population form of Eq. (7), the expected policy gradient on own-memory sessions is:

$$
g_{m} = \frac{1}{s}\nabla_{\theta} J_{b}^{\mathrm{mem}} + \frac{V_{b}}{s}\sum_{t}\mathbb{E}\big[\nabla_{\theta} p_{t}\big],\tag{8}
$$

with $p_t = P_\theta(\omega_t = 1 \mid h_t, m)$ the probability that step $t$ is memory-driven, and its second term vanishes when every $\omega_t = 1$. The first term is outcome learning; the second moves how often the actor acts on the memory—up where the memory pays, down where it does not.

## Empirical Validation / Results

### Main Results (Table 1 excerpt)

For the **Qwen3.8-27B-FP8** actor model:

| Memory layer | Res. (%) ↑ | Steps ↓ | Total tokens/session (K) ↓ | Res./K total tokens ↑ |
|---|---|---|---|---|
| No memory | 21.0 | 35.1 (26.0) | 32.2 (24.7) | 0.653 |
| Full history | 22.4 (+7%) | 32.6 (−7%) | 39.0 (+21%) | 0.574 (−12%) |
| Mem0 | 21.5 (+2%) | 35.9 (+2%) | 68.7 (+113%) | 0.314 (−52%) |
| MemOp | 23.4 (+12%) | 33.5 (−4%) | 50.4 (+57%) | 0.465 (−29%) |
| ReasoningBank | 25.2 (+20%) | 34.3 (−2%) | 67.5 (+110%) | 0.375 (−43%) |
| **CoReMem w/o training** | 23.4 (+11%) | 31.8 (−10%) | 32.4 (0%) | 0.724 (+11%) |
| **CoReMem (full)** | **25.7 (+22%)** | **29.2 (−17%)** | **28.5 (−11%)** | **0.902 (+38%)** |

For the **Qwen3.5-9B** actor model, CoReMem achieves 17.7% resolve (+27%), 72.4 steps (−26%), 79.9K total tokens (−19%), and 0.222 resolve/K total tokens (+46%).

For the **Gemma-4-31B** actor model, CoReMem achieves 14.2% resolve (+24%), 28.5 steps (−5%), 29.3K total tokens (−9%), and 0.486 resolve/K total tokens (+38%).

### Detailed Analysis

**M1: Anchors.** CoReMem packs the most anchors per thousand injected characters (15.8 vs. 14.4 for MemOp, 9.7 for Mem0, 8.5 for ReasoningBank) and achieves the highest used anchors per thousand characters (1.32 vs. 0.92, 0.19, 0.14 respectively).

**M2: Charging steps inside success.** Resolve-only RL lowers steps on 27B from 32.6 to 30.4 and on 9B from 91.9 to 75.5; charging steps inside success (M1+M2) lowers them further to 29.7 and 63.8, with the full method at 29.2 and 72.4.

**M3: Value credit.** Without the value credit, a misplaced memory costs the 27B actor 2.1 steps; with it, the cost shrinks to +0.3 steps. The memory benefit grows from +1.7 to +2.3 resolution points on 27B.

## Theoretical and Practical Implications

### Theoretical Contributions

1. **Prospective valuation framework**: Defines memory value as the change in next-task outcomes caused by the memory (Eq. 2), providing a principled basis for memory design in settings where future tasks are unknown.
2. **Provenance-gated credit assignment**: Shows analytically (Proposition 1) that paying the memory's value to provenance-marked steps creates a policy-gradient push on memory use, which vanishes when the same value is paid to every step.
3. **Cost-internalizing reward design**: The reward (Eq. 4) ensures every resolved session outranks every unresolved one while charging effort only inside success, aligning with the constrained optimization of Eq. (1).

### Practical Implications

- **Memory as anchors**: The finding that value concentrates in locations and checks—not narrative—suggests that efficient cross-task memory should be reference-based rather than summary-based.
- **Actor adaptation is necessary**: The gap between CoReMem w/o training and full CoReMem demonstrates that memory design alone is insufficient; the actor must be trained to act on memory.
- **Token efficiency**: CoReMem achieves lower total tokens (including memory-module calls) than no memory, unlike existing methods that spend more on memory than they save.

## Conclusion

Memory for successive coding tasks is what one task leaves for a task it has not seen. Its worth, measured by what the next task saves when it runs with the memory and without it, concentrates in a few anchors—the locations and checks the next task would otherwise rediscover. A memory built around those anchors lowers the next task's effort on six actor models, and an actor model trained on that value, paid where it acts on them, lowers effort further without lowering resolution.

**Limitations**: Main experiments use Python repositories in the mini-swe-agent harness; post-training covers three open-weight actor models. Other programming languages and larger actor models remain to be tested. Effort is measured by steps and cache-aware tokens, not wall-clock latency or energy use. How memory should be accumulated and refreshed over longer repository histories is future work.

**Future directions**: Testing on additional programming languages and larger models, investigating memory accumulation and refreshment over extended repository histories, and reporting wall-clock latency and energy metrics.

---

_Markdown view of https://picx.dev/p/iqKt76, served by PicX — AI-generated visual whiteboard summaries of research papers._