Summary (Overview)

• Characterizes test-time recurrent scaling: The paper systematically studies LoopLMs (looped language models) beyond their training horizon, showing that additional recurrence can substantially improve reasoning performance but degrades knowledge-oriented tasks.
• Identifies key architectural factors: The effectiveness of recurrence depends on physical depth vs. loop count allocation, placement of non-recurrent boundary layers (Prelude/Coda), and conditioning mechanisms.
• Proposes history-state injection: A novel conditioning method that uses relative differences from intermediate recurrent states, which rescues deep extrapolation performance where conventional initial-state injection fails.
• Demonstrates complementary conditioning: History-state injection combined with timestep conditioning yields the strongest performance gains, outperforming individual variants and the unconditioned baseline under extended inference budgets.
• Provides practical design guidelines: The paper offers actionable insights for designing LoopLMs across variable inference budgets, highlighting that effective depth alone is insufficient to predict behavior.

Introduction and Theoretical Foundation

Background and Motivation: Scaling LLMs has driven substantial capability gains but at significant parameter and memory costs. Looped language models (LoopLMs) offer a parameter-efficient alternative by repeatedly executing a shared stack of Transformer blocks, enabling deeper computation with a smaller memory footprint. Their recurrent structure allows inference depth to be flexibly adjusted by varying the loop count, providing a natural mechanism for scaling test-time computation.

Key Formulation: A looped decoder is formalized as:

$$M_{p,s,c,K} = C \circ F^K \circ P$$

where $P$ is a $p$-block Prelude, $F$ is an $s$-block parameter-shared recurrent core, $C$ is a $c$-block Coda, and $K$ is the loop count during training (the training horizon). At inference, the model executes $r$ loops, where:

• $r < K$: under-unrolling
• $r = K$: training horizon evaluation
• $r > K$: loop extrapolation

The recurrent computation evolves as:

$$h_0 = \tilde{x}; \quad h_{t+1} = F\left(h_t; \gamma_t\right), \quad 0 \leq t < r; \quad y = WC(h_r)$$

where $\gamma_t$ denotes auxiliary conditioning signals (initial state, past recurrent states, or timestep information).

Physical vs. Effective Depth: The paper distinguishes:

• Physical depth: $L_{phys} = p + s + c$ (number of distinct parameterized layers)
• Effective depth: $L(r) = p + r \cdot s + c$ (actual per-token compute)

Research Questions: The paper addresses three fundamental questions:

1. When does recurrence help?
2. Where should recurrence be applied (which computations should be recurrent)?
3. How should recurrence be conditioned?

Methodology

Controlled Experimental Setup: All comparisons match four quantities:

1. Physical depth $L_{phys} = p + s + c$
2. Training effective depth $L(K) = p + K \cdot s + c$
3. Inference effective depth $L(r)$
4. Training pipeline (token budget, optimizer, etc.)

Models and Data:

• Two backbone families: Llama3.1-1B and Qwen3-0.6B
• Trained from scratch on FineWeb-Edu with next-token prediction
• Chinchilla ratio of 20 tokens per parameter
• Muon optimizer for matrix parameters, AdamW for embeddings and non-matrix parameters

Task Categorization:

• Knowledge tasks (fact retrieval, limited multi-step composition): SciQ, ARC-Easy, PIQA
• Reasoning tasks (multi-step inference composition): ARC-Challenge, WinoGrande, OpenBookQA, HellaSwag, CommonsenseQA, ProofWriter, CLUTRR, BBH
• Controlled reasoning-depth axes: ProofWriter (proof depth) and CLUTRR (relation-chain length)

Conditioning Mechanisms Tested:

Initial-state injection (four parameterizations):

$$I^{scalar}_{\phi}(h, h_0) = h + \alpha h_0, \quad I^{channel}_{\phi}(h, h_0) = a \odot h + b \odot h_0$$ $$I^{res}_{\phi}(h, h_0) = (1 + \delta_a) \odot h + b \odot h_0, \quad I^{dense}_{\phi}(h, h_0) = W_h h + W_0 h_0$$

History-state injection (proposed):

$$h_{\ell+1} = F_{\theta}\left(h_{\ell} + \sum_{j=1}^{m_{\ell}} B_j(h_{\ell-j} - h_{\ell})\right), \quad m_{\ell} = \min\{w, \max(\ell-1, 0)\}$$

Timestep conditioning: Three variants—Loop Gating (LG), Branch Gating (BG), and AdaLN—using normalized timestep $t_{\ell} = \ell/L_{infer}$ with step size $\Delta t = 1/L_{infer}$.

Empirical Validation / Results

1. When Does Recurrence Help?

Key Finding 1: Task-dependent extrapolation behavior

• Reasoning: Improves from 28.52 at $L(r)=20$ to 31.49 at $L(r)=40$ (2.97 pp gain at test-time only)
• Knowledge: Decreases from 62.80 to 52.11 over the same range
• The training horizon does not impose a strict ceiling on effective recurrence

Key Finding 2: Harder reasoning instances don't consistently benefit more

• ProofWriter: Extrapolation to $L(r)=24$ boosts accuracy ~7.5 pp at deeper proof depths (3–5)
• CLUTRR: Large gains on short chains (31.6 for length 2, 23.4 for length 3) but smaller gains or declines for longer chains (4–10)

Key Finding 3: Balanced physical depth and loop count allocation matters

• BaseLoop 2×10 (shallow core, many loops): peaks early, degrades under unrolling
• BaseLoop 10×2 (deep core, few loops): remains flat, gains little from extra loops
• BaseLoop 5×4 and 4×5: strongest test-time scaling, continue improving up to $L(r)=40$

Key Finding 4: Recurrent models can surpass non-recurrent upper bounds

• BaseLoop 5×4 reaches 31.88 at $L(r)=30$; BaseLoop 4×5 achieves 31.49 at $L(r)=40$
• Both exceed NonLoop (20×1) reasoning score of 30.25, despite using significantly fewer distinct physical layers

2. Which Computations Should Be Recurrent?

Coda alleviates knowledge decay during under-unrolling: Allocating non-recurrent layers to the Coda (e.g., 0+2×9+2) substantially mitigates degradation, with smaller angular changes and more stable recurrent states.

Prelude-heavy allocations support reasoning extrapolation: Configurations like 2+2×9+0 (four-layer) and 5+5×3+0, 4+5×3+1 (ten-layer) show strong reasoning extrapolation.

Convergence alone doesn't explain useful extrapolation: Beyond the training horizon, loop angular distance and relative update norm decrease, but state variance continues to grow—small per-iteration updates can accumulate and drift states away from training distribution.

3. How Should Recurrence Be Conditioned?

Initial-state injection limitations:

• Dense variants cause catastrophic knowledge collapse during loop extrapolation
• Lightweight variants (Scalar, Channel-wise) avoid breakdown but offer negligible gains
• Static $h_0$ re-injection provides redundant semantics rather than dynamic trajectory guidance

History-state injection benefits:

• With dense parameterization and window $w=1$: rescues from static injection collapse, sustaining accuracy near NonLoop reference
• Window-size sensitivity: $w=2$ or $w=4$ progressively diminishes extrapolation performance
• Low-capacity mappings (Scalar, Channel-wise): negligible gains over BaseLoop

Timestep conditioning results (at $L(r)=56$, Qwen3-0.6B 7×4):

• BG: 38.46 overall, 30.88 reasoning
• LG: 36.91 overall
• AdaLN: 38.01 overall, 29.12 reasoning
• BaseLoop: 36.59 overall

4. When Are Conditioning Mechanisms Composable?

Best pairwise combination: At $D=84$, H+LG reaches 37.30 overall accuracy, exceeding:

• I+H: 36.91
• I+LG: 36.33
• Strongest individual component by 1.16 pp

Geometry analysis reveals complementarity:

• H+LG exhibits 18.6% and 10.4% higher mean interaction over lags 1–6 than I+H and I+LG respectively at $D=84$
• Adding initial-state conditioning to H+LG reduces mean interaction over lags 2–6 by 7.4% and 4.2% at $D=56$ and $D=84$, explaining lack of additive gains

Theoretical and Practical Implications

Theoretical insights:

• Effective depth alone is insufficient to predict LoopLM behavior—the allocation between physical layers and recurrent iterations fundamentally shapes scaling properties
• Transformation magnitude (angular distance, update norm) does not characterize effective iterative refinement; computational interaction strength across loops is a better predictor
• Static conditioning (initial-state injection) provides redundant semantics via residual streams, while dynamic conditioning (history-state) captures trajectory velocity

Practical design guidelines:

1. Task-aware deployment: Use loop extrapolation for reasoning-heavy workloads; avoid for knowledge retrieval tasks
2. Architecture allocation: Balance physical depth and loop count (e.g., 4×5 or 5×4 rather than extreme allocations)
3. Boundary layer placement: Coda layers for robustness to under-unrolling; Prelude-heavy for reasoning extrapolation
4. Conditioning strategy: Channel-wise history-state injection (window $w=2$) with Loop Gating provides the best cost-performance trade-off
5. Avoid over-parameterized conditioning: Dense initial-state injection causes catastrophic knowledge collapse; lightweight scalar/channel-wise variants are preferred

Conclusion

The paper provides a systematic characterization of when, where, and how recurrence is effective in LoopLMs. Key takeaways:

1. Recurrence can scale beyond training horizon for reasoning tasks but degrades knowledge performance, with harder reasoning instances not consistently benefiting more.

2. Architecture matters more than effective depth alone: Balanced physical depth/loop count allocations (4×5, 5×4) show superior extrapolation; Coda layers protect knowledge under under-unrolling; Prelude-heavy designs support reasoning extrapolation.

3. History-state injection is a superior alternative to initial-state injection, capturing dynamic trajectory information through relative state differences. Combined with timestep conditioning (H+LG), it provides complementary gains and robust performance across inference budgets.

4. Future directions: The authors suggest investigating adaptive loop counts, extending to larger model scales, exploring additional conditioning signals, and understanding the theoretical foundations of why balanced configurations extrapolate better.