# What Makes Recurrence Effective in Looped Language Models?

> Test-time loop extrapolation in LoopLMs improves reasoning but degrades knowledge tasks, with balanced depth-loop allocation and history-state injection yielding the strongest gains.

- **Source:** [arXiv](https://arxiv.org/abs/2609.36636)
- **Published:** 2026-10-17
- **Permalink:** https://picx.dev/p/mxeeWF
- **Whiteboard:** https://picx.dev/p/mxeeWF/image

## Summary

## Summary (Overview)

- **Characterizes test-time recurrent scaling**: The paper systematically studies LoopLMs (looped language models) beyond their training horizon, showing that additional recurrence can substantially improve reasoning performance but degrades knowledge-oriented tasks.
- **Identifies key architectural factors**: The effectiveness of recurrence depends on physical depth vs. loop count allocation, placement of non-recurrent boundary layers (Prelude/Coda), and conditioning mechanisms.
- **Proposes history-state injection**: A novel conditioning method that uses relative differences from intermediate recurrent states, which rescues deep extrapolation performance where conventional initial-state injection fails.
- **Demonstrates complementary conditioning**: History-state injection combined with timestep conditioning yields the strongest performance gains, outperforming individual variants and the unconditioned baseline under extended inference budgets.
- **Provides practical design guidelines**: The paper offers actionable insights for designing LoopLMs across variable inference budgets, highlighting that effective depth alone is insufficient to predict behavior.

## Introduction and Theoretical Foundation

**Background and Motivation**: Scaling LLMs has driven substantial capability gains but at significant parameter and memory costs. Looped language models (LoopLMs) offer a parameter-efficient alternative by repeatedly executing a shared stack of Transformer blocks, enabling deeper computation with a smaller memory footprint. Their recurrent structure allows inference depth to be flexibly adjusted by varying the loop count, providing a natural mechanism for scaling test-time computation.

**Key Formulation**: A looped decoder is formalized as:
$$M_{p,s,c,K} = C \circ F^K \circ P$$

where $P$ is a $p$-block Prelude, $F$ is an $s$-block parameter-shared recurrent core, $C$ is a $c$-block Coda, and $K$ is the loop count during training (the *training horizon*). At inference, the model executes $r$ loops, where:
- $r < K$: under-unrolling
- $r = K$: training horizon evaluation
- $r > K$: loop extrapolation

The recurrent computation evolves as:
$$h_0 = \tilde{x}; \quad h_{t+1} = F\left(h_t; \gamma_t\right), \quad 0 \leq t < r; \quad y = WC(h_r)$$

where $\gamma_t$ denotes auxiliary conditioning signals (initial state, past recurrent states, or timestep information).

**Physical vs. Effective Depth**: The paper distinguishes:
- **Physical depth**: $L_{phys} = p + s + c$ (number of distinct parameterized layers)
- **Effective depth**: $L(r) = p + r \cdot s + c$ (actual per-token compute)

**Research Questions**: The paper addresses three fundamental questions:
1. **When** does recurrence help?
2. **Where** should recurrence be applied (which computations should be recurrent)?
3. **How** should recurrence be conditioned?

## Methodology

**Controlled Experimental Setup**: All comparisons match four quantities:
1. Physical depth $L_{phys} = p + s + c$
2. Training effective depth $L(K) = p + K \cdot s + c$
3. Inference effective depth $L(r)$
4. Training pipeline (token budget, optimizer, etc.)

**Models and Data**:
- Two backbone families: Llama3.1-1B and Qwen3-0.6B
- Trained from scratch on FineWeb-Edu with next-token prediction
- Chinchilla ratio of 20 tokens per parameter
- Muon optimizer for matrix parameters, AdamW for embeddings and non-matrix parameters

**Task Categorization**:
- **Knowledge tasks** (fact retrieval, limited multi-step composition): SciQ, ARC-Easy, PIQA
- **Reasoning tasks** (multi-step inference composition): ARC-Challenge, WinoGrande, OpenBookQA, HellaSwag, CommonsenseQA, ProofWriter, CLUTRR, BBH
- **Controlled reasoning-depth axes**: ProofWriter (proof depth) and CLUTRR (relation-chain length)

**Conditioning Mechanisms Tested**:

*Initial-state injection* (four parameterizations):
$$I^{scalar}_{\phi}(h, h_0) = h + \alpha h_0, \quad I^{channel}_{\phi}(h, h_0) = a \odot h + b \odot h_0$$
$$I^{res}_{\phi}(h, h_0) = (1 + \delta_a) \odot h + b \odot h_0, \quad I^{dense}_{\phi}(h, h_0) = W_h h + W_0 h_0$$

*History-state injection* (proposed):
$$h_{\ell+1} = F_{\theta}\left(h_{\ell} + \sum_{j=1}^{m_{\ell}} B_j(h_{\ell-j} - h_{\ell})\right), \quad m_{\ell} = \min\{w, \max(\ell-1, 0)\}$$

*Timestep conditioning*: Three variants—Loop Gating (LG), Branch Gating (BG), and AdaLN—using normalized timestep $t_{\ell} = \ell/L_{infer}$ with step size $\Delta t = 1/L_{infer}$.

## Empirical Validation / Results

### 1. When Does Recurrence Help?

**Key Finding 1: Task-dependent extrapolation behavior**
- **Reasoning**: Improves from 28.52 at $L(r)=20$ to 31.49 at $L(r)=40$ (2.97 pp gain at test-time only)
- **Knowledge**: Decreases from 62.80 to 52.11 over the same range
- The training horizon does not impose a strict ceiling on effective recurrence

**Key Finding 2: Harder reasoning instances don't consistently benefit more**
- ProofWriter: Extrapolation to $L(r)=24$ boosts accuracy ~7.5 pp at deeper proof depths (3–5)
- CLUTRR: Large gains on short chains (31.6 for length 2, 23.4 for length 3) but smaller gains or declines for longer chains (4–10)

**Key Finding 3: Balanced physical depth and loop count allocation matters**
- BaseLoop 2×10 (shallow core, many loops): peaks early, degrades under unrolling
- BaseLoop 10×2 (deep core, few loops): remains flat, gains little from extra loops
- BaseLoop 5×4 and 4×5: strongest test-time scaling, continue improving up to $L(r)=40$

**Key Finding 4: Recurrent models can surpass non-recurrent upper bounds**
- BaseLoop 5×4 reaches 31.88 at $L(r)=30$; BaseLoop 4×5 achieves 31.49 at $L(r)=40$
- Both exceed NonLoop (20×1) reasoning score of 30.25, despite using significantly fewer distinct physical layers

### 2. Which Computations Should Be Recurrent?

**Coda alleviates knowledge decay during under-unrolling**: Allocating non-recurrent layers to the Coda (e.g., 0+2×9+2) substantially mitigates degradation, with smaller angular changes and more stable recurrent states.

**Prelude-heavy allocations support reasoning extrapolation**: Configurations like 2+2×9+0 (four-layer) and 5+5×3+0, 4+5×3+1 (ten-layer) show strong reasoning extrapolation.

**Convergence alone doesn't explain useful extrapolation**: Beyond the training horizon, loop angular distance and relative update norm decrease, but state variance continues to grow—small per-iteration updates can accumulate and drift states away from training distribution.

### 3. How Should Recurrence Be Conditioned?

**Initial-state injection limitations**:
- Dense variants cause catastrophic knowledge collapse during loop extrapolation
- Lightweight variants (Scalar, Channel-wise) avoid breakdown but offer negligible gains
- Static $h_0$ re-injection provides redundant semantics rather than dynamic trajectory guidance

**History-state injection benefits**:
- With dense parameterization and window $w=1$: rescues from static injection collapse, sustaining accuracy near NonLoop reference
- Window-size sensitivity: $w=2$ or $w=4$ progressively diminishes extrapolation performance
- Low-capacity mappings (Scalar, Channel-wise): negligible gains over BaseLoop

**Timestep conditioning results** (at $L(r)=56$, Qwen3-0.6B 7×4):
- BG: 38.46 overall, 30.88 reasoning
- LG: 36.91 overall
- AdaLN: 38.01 overall, 29.12 reasoning
- BaseLoop: 36.59 overall

### 4. When Are Conditioning Mechanisms Composable?

**Best pairwise combination**: At $D=84$, H+LG reaches 37.30 overall accuracy, exceeding:
- I+H: 36.91
- I+LG: 36.33
- Strongest individual component by 1.16 pp

**Geometry analysis reveals complementarity**:
- H+LG exhibits 18.6% and 10.4% higher mean interaction over lags 1–6 than I+H and I+LG respectively at $D=84$
- Adding initial-state conditioning to H+LG reduces mean interaction over lags 2–6 by 7.4% and 4.2% at $D=56$ and $D=84$, explaining lack of additive gains

## Theoretical and Practical Implications

**Theoretical insights**:
- Effective depth alone is insufficient to predict LoopLM behavior—the allocation between physical layers and recurrent iterations fundamentally shapes scaling properties
- Transformation magnitude (angular distance, update norm) does not characterize effective iterative refinement; computational interaction strength across loops is a better predictor
- Static conditioning (initial-state injection) provides redundant semantics via residual streams, while dynamic conditioning (history-state) captures trajectory velocity

**Practical design guidelines**:
1. **Task-aware deployment**: Use loop extrapolation for reasoning-heavy workloads; avoid for knowledge retrieval tasks
2. **Architecture allocation**: Balance physical depth and loop count (e.g., 4×5 or 5×4 rather than extreme allocations)
3. **Boundary layer placement**: Coda layers for robustness to under-unrolling; Prelude-heavy for reasoning extrapolation
4. **Conditioning strategy**: Channel-wise history-state injection (window $w=2$) with Loop Gating provides the best cost-performance trade-off
5. **Avoid over-parameterized conditioning**: Dense initial-state injection causes catastrophic knowledge collapse; lightweight scalar/channel-wise variants are preferred

## Conclusion

The paper provides a systematic characterization of when, where, and how recurrence is effective in LoopLMs. Key takeaways:

1. **Recurrence can scale beyond training horizon** for reasoning tasks but degrades knowledge performance, with harder reasoning instances not consistently benefiting more.

2. **Architecture matters more than effective depth alone**: Balanced physical depth/loop count allocations (4×5, 5×4) show superior extrapolation; Coda layers protect knowledge under under-unrolling; Prelude-heavy designs support reasoning extrapolation.

3. **History-state injection is a superior alternative** to initial-state injection, capturing dynamic trajectory information through relative state differences. Combined with timestep conditioning (H+LG), it provides complementary gains and robust performance across inference budgets.

4. **Future directions**: The authors suggest investigating adaptive loop counts, extending to larger model scales, exploring additional conditioning signals, and understanding the theoretical foundations of why balanced configurations extrapolate better.

---

_Markdown view of https://picx.dev/p/mxeeWF, served by PicX — AI-generated visual whiteboard summaries of research papers._