# Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization

> Latent reasoning models discover a generalizable recurrent forward-search algorithm, while token-based methods like CoT rely on local shortcuts that fail out-of-distribution.

- **Source:** [arXiv](https://arxiv.org/abs/2609.35643)
- **Published:** 2026-11-07
- **Permalink:** https://picx.dev/p/Ql5S8x
- **Whiteboard:** https://picx.dev/p/Ql5S8x/image

## Summary

## Summary (Overview)

- **Core Question**: Do different forms of "thinking" in language models—token-based traces (CoT), pause tokens, and latent-space reasoning—rely on the same underlying computational mechanisms?

- **Key Finding**: Strong in-distribution (ID) performance does not guarantee depth generalization. Vanilla, CoT, and Pause Token models rely on local graph shortcuts and fail on out-of-distribution (OOD) problems with longer hops, while latent reasoning models generalize effectively.

- **Mechanistic Discovery**: The bottleneck-latent model spontaneously discovers a **recurrent forward-search algorithm**—implemented as a sparse circuit with one attention head retrieving graph relations, an MLP updating reachability state, and multiple attention heads performing candidate matching.

- **Surprising Result**: Chain-of-Thought, despite explicit step-by-step proof supervision, performs *worst* on OOD generalization, demonstrating that training on reasoning traces does not guarantee genuine reasoning.

- **Methodological Contribution**: The paper introduces a controlled comparison of five model variants sharing the same backbone, dataset, and training budget, isolating the effect of the "thinking interface" on learned computation.

---

## Introduction and Theoretical Foundation

The paper addresses a fundamental question in mechanistic interpretability: **do different forms of intermediate computation in LLMs converge to the same underlying solution, or do they induce mechanistically distinct algorithms?**

**Background context:**
- CoT prompting ("Let's think step by step") improves reasoning performance (Kojima et al., 2022; Wei et al., 2022)
- Reasoning traces can be partially removed without harming answers (Lanham et al., 2023), and even meaningless filler tokens help (Pfau et al., 2024)
- Latent reasoning (Coconut-style, Hao et al., 2025) feeds high-dimensional vectors directly into the model instead of tokens
- Prior work (Zhu et al., 2025) theorized latent thoughts can encode multiple search frontiers in superposition, but causal evidence has been contested (Aswal et al., 2026)

**Theoretical foundation**: The paper leverages symbolic reasoning tasks (multi-hop graph reachability) as a controlled probe. The task has a well-defined algorithmic solution—forward reachability propagation—making it possible to compare model-internal representations against ground-truth algorithmic states.

**Key design principle**: By training all variants from random initialization (unlike prior work using pretrained models), the paper ensures that any learned computation comes entirely from training on the task, eliminating confounds from pretraining knowledge.

---

## Methodology

### Task: ProsQA-Ext
- Each sample $(x, y)$ describes a directed acyclic graph (DAG) $G = (V, E)$ with premises of form "A is B."
- Query asks whether root $r$ can reach candidate $c_0$ or $c_1$ at shortest distance $H$
- **Training set**: $H \in \{3, \ldots, 6\}$; **OOD validation**: $H \in \{7, \ldots, 12\}$
- Graph generation carefully controls for superficial features (~50% chance baseline)

### Model Variants (all GPTNeoX backbone: 4 layers, hidden size 256, FFN 768)

| Variant | Thinking Mechanism | Readout |
|---------|-------------------|---------|
| **Direct** | None (answer immediately) | Standard |
| **Chain-of-Thought** | Generates proof tokens first | Supervised on proofs + answers |
| **Pause-token** | K=6 identical learnable embeddings | Standard |
| **Full-latent** | Recurrent latent vectors (Coconut-style) | Retains prompt cache |
| **Bottleneck-latent** | Recurrent latent vectors | Re-encodes only $z_{1:K}$ as prefix |

The latent recurrence is formalized as:

$$
z_{1} = N_{\theta}\Big(h_{n}^{(L)}\Big), \quad z_{t+1} = N_{\theta}\Big(h_{n+t}^{(L)}\Big), \qquad 1 \leq t < K, \tag{1}
$$

where each vector enters the first block as $h_{n+t}^{(0)} = z_t$, and $n = |x|$.

### Analysis Methods
1. **Representational Similarity Analysis (RSA)**: Correlates model representation dissimilarities with algorithmic forward-propagation frontier dissimilarities
2. **Matched-pair interventions**: Manipulate local graph features (candidate in-degree, successor degrees) to test shortcut reliance
3. **Latent-state transplantation**: Swap latent states between matched inputs to probe causal influence
4. **Circuit pruning**: Greedily remove components while preserving ≥90% output consistency and $R^2 > 0.8$ for candidate logit margin

---

## Empirical Validation / Results

### ID vs. OOD Performance

- **ID (H=3–6)**: All variants except Direct achieve near-perfect or high accuracy
- **OOD (H=7–12)**: Bottleneck-latent > Full-latent > Pause-token > Direct > CoT

> *"Models that perform similarly on the training range can generalize very differently beyond it."*

### Representational Alignment

- **Bottleneck-latent** shows the highest and most consistent **diagonality** score—indicating representations track forward reachability propagation step-by-step
- Direct and Pause-token show nearly no sequential alignment
- CoT shows some alignment (consistent with proof supervision) but still fails OOD

### Shortcut Reliance in Non-Latent Models

| Variant | Candidate in-degree effect | Successor degree effect |
|---------|---------------------------|------------------------|
| Direct | **Strong** | Minimal |
| Pause-token | **Strong** | Minimal |
| Chain-of-Thought | Minimal | **Strong** |
| Full/Bottleneck-latent | Weak | Weak |

CoT explicitly prefers successors with lower in-degree minus out-degree when selecting its first proof step.

### Recurrent Search Evidence (Bottleneck-latent)

The transplantation experiments reveal a clear **upper-triangular pattern**: shallow connectivity swaps affect latent states early, deep swaps affect them later—consistent with step-by-step reachability propagation.

**K-sensitivity**: Bottleneck-latent retains performance under $K = 5, 6, 7$, while Full-latent degrades at $K=5$ and Pause-token at $K=7$, supporting a "soft" (non-strict) iteration hypothesis.

### Sparse Circuit Localization

Pruning identified **8 of 20 recurrent components** preserving:
- 91.9% output consistency (full model)
- 90.9% consistency on held-out OOD samples (vs. 52.7% when removed, 55.8% for random subsets)
- Peak causal transfer effect of 63.9% (vs. 69.9% full model)

**Functional roles identified:**
- **L4H1**: "Address–content" organization—keys encode premise sources (LHS), values encode destinations (RHS); LHS swaps affect keys, RHS swaps affect values
- **L4MLP**: Acts as a "filter" with the residual stream, integrating retrieved information into the next recurrent state
- **L4H1/H2/H4**: Candidate matching—Q/K interventions show evidence depends on match between current recurrent state and candidates' graph context

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Thinking interfaces induce mechanistically distinct solutions**: The paper provides direct evidence that different forms of intermediate computation do not converge to the same algorithm, even at matched performance levels.

2. **Latent recurrence enables algorithmic discovery**: Without explicit supervision or reward signals, the bottleneck-latent model spontaneously discovers a forward-search algorithm—suggesting that full information flow across steps is sufficient for discovering generalizable computation.

3. **CoT supervision can be counterproductive for generalization**: Explicit step-by-step proof training led the model to rely on local degree heuristics rather than genuine reachability computation, performing worst on OOD problems.

4. **"Soft" iteration as a computational primitive**: The latent model handles more than one graph hop per recurrent step, suggesting a flexible, non-strict iterative mechanism rather than a hard for-loop.

### Practical Implications

- **Evaluation methodology**: OOD testing is essential before mechanistic analysis—ID performance alone is insufficient to infer reasoning quality
- **Architecture design**: Bottleneck constraints (removing prompt cache access) may *encourage* more robust recurrent computation
- **Interpretability**: The sparse circuit localization demonstrates that latent reasoning in small models can be reduced to interpretable components, analogous to circuit analysis in vision models

---

## Conclusion

The paper demonstrates that **different thinking mechanisms can learn fundamentally different computational solutions at similar in-distribution performance**. The key takeaways:

1. **Latent reasoning models** (especially bottleneck-latent) discover a reusable forward-search algorithm that generalizes beyond training depth, implemented as a sparse recurrent circuit
2. **Token-based approaches** (Direct, CoT, Pause-token) rely on local graph shortcuts, which suffice in-distribution but fail OOD
3. **The "thinking interface" matters**: Full information flow across latent steps enables algorithmic discovery that explicit trace supervision does not

**Future directions** identified by the authors:
- Whether looped transformers (weight-tied blocks) formalize the same iterative computation discovered here
- Whether conclusions extend to larger pretrained LLMs and more naturalistic reasoning tasks
- The role of recurrence and bottleneck constraints in larger-scale systems

The paper's controlled experimental design—matching backbone, data, and budget while varying only the thinking interface—provides a template for isolating mechanistic differences in reasoning architectures.

---

_Markdown view of https://picx.dev/p/Ql5S8x, served by PicX — AI-generated visual whiteboard summaries of research papers._