# What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

> Recursive reasoning performance is driven primarily by optimization choices—gradient horizon, physical batch size, and stabilized state updates—rather than architectural hierarchy, yielding a 13.6M-parameter model that doubles OOD accuracy on arithmetic.

- **Source:** [arXiv](https://arxiv.org/abs/2609.39967)
- **Published:** 2026-11-14
- **Permalink:** https://picx.dev/p/15ZtUH
- **Whiteboard:** https://picx.dev/p/15ZtUH/image

## Summary

## Summary (Overview)

- **Controlled ablation study**: The paper systematically isolates architectural and optimization factors in recursive reasoning models (HRM, TRM, URM) under a unified pipeline across six algorithmic domains, revealing that optimization choices account for a large fraction of performance variation.
- **Key recipe for stable recursion**: An intermediate gradient horizon $(K_L, K_H) = (2, 2)$, large physical batches (up to 8192), and controlled recurrent-state updates (bounded updates + learned gating + normalization) yield stable and generalizable recursive reasoning.
- **No hierarchy needed**: Explicit high/low-level parameter separation is not a universal source of improvement; shared recurrent modules remain competitive across all domains.
- **Strong empirical results**: A 13.6M-parameter model achieves 71.16% on Arithmetic OOD (up from 36.20% for the strongest baseline), 98.41% on Sudoku, and 59.50% pass@2 on ARC-AGI-1.
- **Scaling limits**: Additional capacity (larger recurrent blocks, more heads) and test-time computation provide domain-dependent rather than universal benefits.

---

## Introduction and Theoretical Foundation

**Background and Motivation.** Recursive reasoning models repeatedly apply a small shared Transformer block to a latent state, trading parameter count for computational depth. Models like HRM (27M params), TRM (7M), and URM (7M) solve symbolic algorithmic tasks with only a few million parameters. However, these models change *architecture, gradient propagation, optimization, and regularization simultaneously*, making it impossible to attribute improvements to individual design choices.

**Core Problem.** Repeated application of shared parameters creates optimization challenges distinct from standard feed-forward Transformers:
- Small errors in recurrent updates accumulate over depth
- Gradient propagation through long recurrent trajectories is unstable
- The interplay between effective depth, gradient flow, and generalization is poorly understood

**Theoretical Basis.** The work builds on the nested recurrence formulation introduced by HRM, where two timescales (low and high) of recurrent states interact:
- Low-level state updates: $z_L \leftarrow L(z_L + z_H + x)$
- High-level update: $z_H \leftarrow H(z_H + z_L)$

The authors argue that recursive reasoning should be viewed as **an optimization problem as much as an architectural one** — the effectiveness of recurrence depends on *how information and gradients propagate through the recurrent trajectory*.

---

## Methodology

### 3.1 Data Domains

Six algorithmic domains are used with a unified interface (padded token sequences → complete target prediction, no intermediate supervision):

| Domain | Task | OOD Split |
|--------|------|-----------|
| **Sudoku** | 9×9 puzzle completion (81 tokens) | Standard split |
| **Maze** | 30×30 path finding (900 tokens) | Standard split |
| **Game of Life** | Predict state after $k$ updates on up to 17×17 grids | 15% held-out patterns + extrapolation 1, 2, 3, 10 steps beyond training horizon |
| **Arithmetic** | Recover operator sequence from RPN expressions (3–8 operands, up to $4^7$ assignments) | Split by operand multiset + target range [0, 101] vs [102, 201] |
| **ARC-AGI-1** | Abstract rule induction | Combined with ConceptARC |
| **ARC-AGI-2** | Abstract rule induction | Standard split |

### 3.2 Controlled Recurrent-State Updates

To address instability from repeated transformations, the authors introduce a three-part stabilization mechanism:

**Update bounding** (constrain candidate update magnitude relative to current state):
$$\tilde{z}_L = L(z_L + z_H + x), \quad \Delta_L = \tilde{z}_L - z_L, \quad r = \frac{\|\Delta_L\|_2}{\|z_L\|_2 + \epsilon}, \quad \hat{\Delta}_L = \frac{\Delta_L}{\max(r/\tau, 1)}$$

**Learned gating** (control how much of the bounded update is applied):
$$\alpha = \sigma\left(w^T(z_H + z_L + x) + b\right), \quad z_L' = \text{Norm}(z_L + \alpha \odot \hat{\Delta}_L)$$

where ⊙ is element-wise multiplication with α broadcast over the hidden dimension.

**Recurrence-consistent regularization**: One dropout mask reused across all recurrent cycles and ACT steps (rather than independent masks), plus norm-scaled Gaussian noise on input embeddings and recurrent states.

### 3.3 Training Configuration

- **Gradient horizon**: Gradients propagate only through the final $K_H = K_L = 2$ cycles (truncated backpropagation)
- **Optimizer**: Adam-atan2, global gradient clipping at 1, EMA weights for evaluation
- **Physical batches**: Large (up to 8192), rather than equivalent gradient accumulation
- **Final model**: Shared 4-layer Transformer block, hidden size 512, (H_cycles, L_cycles) = (4, 2)

**Final architecture summary** (Table 1 comparison):

| Setup | HRM | TRM | URM | Ours |
|-------|-----|-----|-----|------|
| Parameters | 27M | 7M | 7M | 13.6M |
| Modules | 2 (H+L) | 1 shared | 1 shared | 1 shared |
| Layers | 4+4 | 2 | 2 | 4 |
| Hidden size | 512 | 512 | 512 | 512 |
| Cycles (H,L) | (2,2) | (3,6) | (3,6) | (4,2) |
| State stabilization | — | — | — | Bounded + gated + norm. |

---

## Empirical Validation / Results

### 4.1 Main Results (Table 2)

| Metric | Dense | Qwen3-30B | HRM | TRM | URM | **Ours** |
|--------|-------|-----------|-----|-----|-----|---------|
| Arithmetic ID | 78.50 | 0.10 | 92.00 | 98.80 | 95.60 | **96.34** |
| **Arithmetic OOD** | 7.49 | 0.00 | 13.80 | 36.20 | 28.50 | **71.16** |
| Game of Life ID | 8.17 | 0.00 | 5.00 | 7.11 | 55.42 | **66.08** |
| Game of Life OOD | 7.54 | 0.00 | 0.30 | 7.14 | 55.26 | **65.80** |
| Maze | 8.00 | 0.20 | 71.10 | 80.00 | 80.20 | **84.70** |
| Sudoku | 18.30 | 0.00 | 50.00 | 87.40 | 77.60 | **98.41** |
| ARC-AGI-1 pass@1 | 30.75 | 25.00 | 32.00 | 40.00 | 49.75 | **53.00** |
| ARC-AGI-1 pass@2 | 33.50 | 31.50 | 40.30 | 44.60 | 53.25 | **59.50** |
| ARC-AGI-2 pass@1 | 1.67 | 0.83 | 3.33 | 2.50 | 5.83 | **10.83** |
| ARC-AGI-2 pass@2 | 1.67 | 0.83 | 4.17 | 2.50 | 7.50 | **11.67** |

**Key finding**: The largest improvement is on **Arithmetic OOD**: accuracy increases from 36.20% (strongest recursive baseline) to 71.16% — nearly a 2× improvement.

### 4.2 Gradient Horizon Ablation (Figure 4)

Varying only the backward horizon $(K_L, K_H)$ with forward recursion fixed at $L_{cycles}=2$, $H_{cycles}=4$:

- **Intermediate optimum**: $(K_L, K_H) = (2,2)$ achieves highest OOD accuracy (71.16%)
- $(1,3)$ reaches highest ID accuracy (99.60%) but generalizes poorly (17.88% OOD)
- **Crucially**: $(1,3)$ and $(2,2)$ differentiate the *same number* of recurrent applications but produce sharply different OOD behavior → both the **extent and placement** of gradient flow matter.

### 4.3 Physical Batch vs. Gradient Accumulation (Figure 5)

At fixed effective batch size 8192 on Game of Life:

| Physical Batch | Accum. Steps | OOD Accuracy | Mean Pre-Clip Grad Norm |
|---------------|--------------|--------------|------------------------|
| 8192 | 1 | **63.16%** | 0.38 |
| 4096 | 2 | — | — |
| 2048 | 4 | 37.71% | 65.46 |

Equal effective batch size does **not** produce equivalent training dynamics — physical batch size is an optimization parameter in its own right.

### 4.4 Stabilization Ablation (Table 3)

Cumulative addition of stabilization mechanisms (mean ± std over 3 seeds):

| Configuration | GoL ID | GoL OOD | Sudoku |
|---------------|--------|---------|--------|
| Base | 61.95 ± 3.32 | 63.87 ± 2.90 | 97.29 ± 1.06 |
| + gate | 64.24 ± 2.59 | 65.89 ± 2.77 | 98.41 ± 0.25 |
| + grad clip | 62.41 ± 1.56 | 63.87 ± 1.54 | 98.50 ± 0.20 |
| + dropout | 61.82 ± 0.34 | 62.09 ± 0.41 | 98.36 ± 0.25 |
| **+ noise (final)** | **66.08 ± 0.11** | **65.80 ± 0.13** | **98.41 ± 0.21** |

**Critical insight**: The stabilization recipe's main effect is turning a *seed-sensitive* optimization problem into a *reproducible* one — standard deviation drops from 3.32 to 0.11 points on Game of Life ID while accuracy rises.

### 4.5 Hierarchy and Scaling Ablations (Figure 6)

- **Shared vs. separate modules**: No consistent winner. Shared performs better on Maze and ARC-AGI-1; separate provides small gains on Sudoku and Game of Life. Explicit hierarchy is **not necessary**.
- **Test-time scaling**: Additional ACT budget helps most on Arithmetic and Sudoku; gains are limited on Game of Life, Maze, and ARC-AGI.
- **Capacity scaling**: Increasing recurrent block from 4 to 8 layers, increasing hidden size, or adding attention heads yields no consistent accuracy gains.

---

## Theoretical and Practical Implications

**Theoretical Contributions:**

1. **Recursive reasoning is primarily an optimization problem**. The performance gap between models is driven more by *how recurrence is trained and stabilized* than by architectural choices. This reframes the research agenda for recursive models.

2. **Non-monotonic gradient horizon**. The finding that $(2,2)$ and $(1,3)$ — which differentiate the same number of recurrent applications — produce starkly different OOD performance reveals that gradient *placement* matters as much as gradient *length*. This suggests a nuanced interaction between credit assignment structure and generalization in recurrent computation.

3. **Physical batch size as a first-class optimization parameter**. The observation that physical batch size affects optimization beyond effective batch size (gradient accumulation degrades performance) challenges common assumptions in distributed training literature.

4. **Stabilization enables reproducibility**. The dramatic reduction in seed sensitivity (std from 3.32 → 0.11 points) suggests that controlled recurrent-state updates tame a fundamentally unstable optimization landscape.

**Practical Implications:**

- **Recipe for stable recursive reasoning**: Intermediate gradient horizon (2,2), large physical batches, bounded + gated + normalized state updates, recurrence-consistent dropout, and relative noise.
- **Architectural minimalism**: A single shared recurrent block suffices — explicit hierarchical decomposition is unnecessary, reducing model complexity.
- **Component-wise optimization**: The paper demonstrates that "AI systems can be improved by optimizing their components one at a time" — a broadly applicable methodology.
- **Domain-dependent scaling**: Practitioners should not expect universal gains from adding capacity or test-time compute; these should be tuned per domain.

---

## Conclusion

**Main Takeaways:**

1. **Three critical optimization factors** govern stable recursive reasoning: (a) an intermediate gradient horizon (neither aggressive truncation nor full backpropagation), (b) large physical batches (not gradient accumulation), and (c) controlled recurrent-state updates (bounded, gated, normalized).

2. **Architectural simplicity suffices**: Explicit hierarchy, larger recurrent blocks, and additional test-time computation provide domain-dependent rather than universal gains.

3. **Strong empirical validation**: The resulting 13.6M-parameter model outperforms all evaluated recursive baselines on nearly all metrics, with especially dramatic gains on OOD generalization (Arithmetic OOD: 36.2% → 71.2%).

**Future Directions:**

- The finding that gradient *placement* matters (not just length) opens questions about optimal credit assignment structures in recurrent computation.
- The domain-dependent scaling behavior suggests a need for task-aware configuration rather than universal recipes.
- The success of stabilization mechanisms invites further investigation into other forms of trajectory regularization.
- The large OOD gains raise the question: can similar optimization-focused improvements be achieved in larger-scale recursive language models?

**Broader Message**: Recursive reasoning models should be viewed as *optimization problems as much as architectural ones*. The effectiveness of recurrence depends critically on how information and gradients are propagated through the recurrent trajectory — a principle that likely extends beyond the small algorithmic solvers studied here.

---

_Markdown view of https://picx.dev/p/15ZtUH, served by PicX — AI-generated visual whiteboard summaries of research papers._