Summary (Overview)

• Introduces Test-Time Memory Evolution (TTME), a plug-and-play, non-parametric alternative to Test-Time Training (TTT) for scientific discovery tasks.
• TTME maintains two complementary external memory modules—Guideline Memory (operational lessons) and Strategy Memory (reusable algorithmic ideas)—to guide a fixed-weight LLM without updating model parameters.
• Outperforms TTT on all five mathematical tasks across two discovery frameworks (TTT-Discover, ThetaEvolve) and three models (8B–120B parameters).
• Reduces computational cost by 82.8–84.6% in FLOPs and 47.7–58.4% in runtime relative to TTT, while achieving better discovery outcomes.
• Demonstrates that self-evolving external memory is a more efficient and effective adaptation mechanism than online weight updates for scientific discovery.

Introduction and Theoretical Foundation

Scientific discovery is modeled as an iterative process: propose hypotheses → evaluate → use evidence to guide further investigation. Recent Test-Time Training (TTT) frameworks (e.g., TTT-Discover, ThetaEvolve) combine iterative search with test-time reinforcement learning, enabling open-weight LLMs to outperform inference-only frontier models. However, TTT incurs significant computational cost from online weight updates, limiting scalability.

Core hypothesis: Since LLMs already possess broad foundational knowledge, continually distilling and organizing search experience in external memory may be sufficient to improve subsequent discovery—without internalizing experience through weight updates.

The paper formalizes two adaptation mechanisms:

$$\begin{array}{r l} \text{Weight adaptation:} & \theta_ {t + 1} \leftarrow U _ {\mathrm{RL}} (\theta_ {t}, \mathcal {D} _ {t}), \\ \text{Memory adaptation:} & \mathcal {M} _ {t + 1} \leftarrow U _ {\mathrm{mem}} (\mathcal {M} _ {t}, \mathcal {D} _ {t}), \qquad \theta_ {t} = \theta_ {0}. \end{array}\tag{2}$$

where $U_{RL}$ updates model parameters, $U_{mem}$ updates external memory, $\theta_t$ is the model parameter, and $\mathcal{M}_t$ is the external memory. The goal is to maximize the best valid score:

$$s^{\star} = \max_{\substack{x\in \mathcal{X}_{\text{eval}}\\ v(x) = 1}}s(x).\tag{1}$$

Methodology

Guideline Memory ($\mathcal{L}_t$)

• Distills execution and evaluation feedback into compact operational guidelines (when applicable, failure/improvement explanation, recommended action).
• Updated every $K$ search steps via two stages: Distill (propose new guidelines from outcomes) and Maintain (match, merge, or add entries; enforce capacity and recency).

$$\mathcal {G} _ {t} \leftarrow \operatorname{Distill} (r _ {t}, \mathcal {L} _ {t}), \qquad \mathcal {L} _ {t + 1} \leftarrow \operatorname{Maintain} (\mathcal {L} _ {t}, \mathcal {G} _ {t}, t).\tag{3}$$

Strategy Memory ($\mathcal{S}_t$)

• Extracts algorithmic ideas from score-improving modifications (where modified program beats its parent by >$10^{-5}$).
• Groups related experience notes into strategy families; consolidates new notes into existing or new entries.

$$\mathcal {E} _ {t} \leftarrow \operatorname{Extract} (\mathcal {D} _ {t} ^ {+}), \qquad \mathcal {S} _ {t + 1} \leftarrow \operatorname{Consolidate} (\mathcal {S} _ {t}, \mathcal {E} _ {t}).\tag{4}$$

Strategy-Guided Search

For each parent $p_i^{(t)}$ with context $C_i^{(t)}$:

1. Select up to $N$ distinct strategies based on improvement potential (Eq. 5).
2. Adapt selected strategies into parent-specific modification instructions, or Propose new ideas when none selected (Eq. 6).
3. Generate candidates via two branches:
   • Direct Generation (fraction $\alpha = 1/2$ of budget $B$): uses parent context + Guideline Memory (Eq. 7).
   • Strategy-Guided Generation: uses parent context + Guideline Memory + modification instruction (Eq. 8).

The full procedure is outlined in Algorithm 1, preserving each framework's parent-selection and archive-update rules.

Empirical Validation / Results

Performance Across Frameworks and Models

Table 1: TTT-Discover framework (50 search steps, results for w/ TTT from Yuksekgonul et al.)

Table 2: ThetaEvolve framework (65 search steps, results for w/ TTT from Wang et al.)

Key finding: TTME outperforms TTT in all evaluated settings, and also surpasses the fixed-weight baseline (w/o TTT).

Efficiency Gains

Table 3: Runtime and computation under TTT-Discover with Qwen3-8B

Ablation Studies

Table 4: Combining TTME with TTT (TTT-Discover, Qwen3-8B)

Adding TTT to TTME worsens results, supporting TTME as a standalone alternative.

Table 5: Memory component ablation (TTT-Discover, gpt-oss-120b)

Both memory components contribute; removing either degrades performance.

Search Behavior Analysis

• Validity rate: Both TTT and TTME improve implementation validity over baseline; TTME achieves better scores with similar validity, suggesting more effective idea generation.
• Idea combination: TTME selects combinations of previously explored ideas more frequently than TTT (AC1: 10.2% vs 8.2%; AC2: 24.7% vs 18.6%), and more frequently selects score-improving combinations (AC1: 5.4% vs 4.8%; AC2: 17.1% vs 13.3%).

Theoretical and Practical Implications

• Efficiency vs. effectiveness trade-off: TTME demonstrates that non-parametric adaptation (external memory) can outperform parametric adaptation (weight updates) while being substantially cheaper—challenging the assumption that online RL is necessary for test-time improvement.
• Scalability: The significant reduction in FLOPs and runtime makes TTME more feasible for scaling to larger models and more complex discovery tasks where TTT's computational cost becomes prohibitive.
• Modularity: As a plug-and-play method, TTME can be integrated into existing discovery frameworks without modifying their search logic, offering a practical drop-in replacement for TTT.
• Knowledge organization: The two-memory design (operational guidelines vs. algorithmic strategies) provides a principled way to structure search experience, with each component contributing complementary benefits.

Conclusion

The paper establishes Test-Time Memory Evolution (TTME) as an effective and computationally efficient alternative to Test-Time Training for scientific discovery. By distilling search experience into external Guideline and Strategy memories, TTME guides fixed-weight LLMs to achieve better discovery outcomes than TTT across two frameworks, three models, and five mathematical tasks—while reducing computational cost by up to 84.6% in FLOPs and 58.4% in runtime. The analysis reveals that TTME more frequently combines and builds upon productive ideas from previous search attempts. Future directions may include scaling TTME to larger models, applying it to more diverse discovery tasks, and exploring additional memory architectures or update mechanisms.