Full text not available for this paper
Summary (Overview)
- Key Finding: Completing the full preset latent horizon in latent Chain-of-Thought (CoT) reasoning can actually degrade reasoning performance, as better reasoning outcomes often emerge before the preset horizon is exhausted but fail to be preserved.
- Three-Stage Evolution: Latent CoT evolves through three distinct functional stages: frontier advancement (new reasoning candidates form), candidate organization (existing candidates are reorganized), and answer revision (later steps mainly revise existing candidates, potentially overturning gold answers).
- Method: The authors introduce a lightweight estimator that predicts remaining reasoning progress from the observed latent trajectory and adaptively adjusts the latent horizon during inference—without retraining the LLM.
- Results: Adaptive latent inference improves reasoning accuracy by up to 7.2 percentage points while reducing latent computation by up to 29.0% across CODI, Coconut, and SIM-CoT latent reasoners on GSM8K, SVAMP, and MultiArith benchmarks.
- Theoretical Foundation: The paper provides formal results (Proposition 4.1, Theorem 4.2) characterizing optimal estimation of remaining progress and monotone allocation of additional computation based on remaining progress.
Introduction and Theoretical Foundation
Background and Motivation
Latent reasoning (Deng et al., 2023; Hao et al., 2024) has emerged as an alternative to explicit Chain-of-Thought (CoT) reasoning (Wei et al., 2022). Unlike explicit CoT, which externalizes intermediate reasoning as discrete language tokens, latent CoT carries intermediate reasoning directly in continuous representations, enabling:
- Maintenance of multiple reasoning alternatives (Zhu et al., 2026b)
- Compression of explicit reasoning into continuous states (Cheng & Van Durme, 2024; Shen et al., 2025)
Existing latent reasoners typically execute a fixed number of latent steps (preset latent horizon ) before generating an explicit answer. The intuitive assumption is that a longer latent horizon should provide more opportunity for reasoning (Goyal et al., 2024).
Key Observation
The paper challenges this intuition by showing that better reasoning outcomes can emerge before the preset horizon is exhausted and fail to be preserved at the full horizon (Figure 1). This suggests that reasoning benefits do not simply accumulate as latent CoT becomes deeper.
Problem Formulation
Latent CoT: Let index latent computation steps, where is the preset latent horizon. The reasoning state after the -th latent step is . Given input and preceding states , the model induces:
After latent steps, the answer distribution is , with task reward scoring generated answers.
Adaptive Latent Horizon: Let denote the realized latent horizon, with the answer generated as . The goal is to determine from latent computation available during inference, where continuation at step depends only on and prefix states .
Methodology
1. Identifying Three Functional Stages
Using the ProsQA benchmark (Hao et al., 2024) with explicit reasoning graphs, the authors analyze the symbolic Coconut model (Zhu et al., 2026b) by decoding reasoning candidates at each latent step and tracking probability changes:
| Stage | Description | Characteristics |
|---|---|---|
| Frontier Advancement | Earlier evolution forms new task-relevant reasoning states | Candidates at smaller graph depths receive support first; support moves through intermediate reasoning before answers emerge |
| Candidate Organization | Later computation reorganizes already-emerged candidates | Fewer new candidates enter high-support region; existing candidates continue receiving support |
| Answer Revision | Final stage mainly revises existing answer candidates | Support shifts among existing candidates; can weaken gold candidate support and overturn established answers |
2. Empirical Tests of Reasoning Gains
Test 1 (Frontier Stage): Interrupting ongoing latent CoT at the frontier stage by inserting a single explicit realization (decoding current state to its top token and adding it to the causal context) markedly delays subsequent progression toward the gold answer. A matched control with comparable perturbations preserves candidate progression.
Test 2 (Revision Stage): Later computation recovers the gold answer in some cases but also overturns already-established gold results, leaving little net gain. Some later revisions allow gains reached earlier to be lost.
3. Remaining Reasoning Progress Estimation
Current reasoning value after latent steps:
Remaining reasoning progress (comparing current frontier to best attainable within full horizon):
Training the estimator: A lightweight estimator is trained on a causal trajectory representation (capturing short-range changes in predictive statistics and hidden states) to predict remaining progress:
Horizon adaptation: Each estimated remaining progress is expressed as a relative position in a frozen reference distribution:
Additional latent computation allocated after step :
where is a bounded, non-decreasing continuation rule.
4. Theoretical Results
Proposition 4.1 (Causal Estimation of Remaining Progress): Under population squared risk , the optimal estimator satisfies:
Theorem 4.2 (Monotone Allocation by Remaining Progress): Let have increasing differences in . For trajectories with remaining progress , consider integer allocations maximizing subject to shared compute budget . Then there exists an optimal allocation such that implies .
Empirical Validation / Results
5.1 Adaptive Latent Inference Results
Table 1: Adaptive latent CoT inference (accuracy %)
| GSM8K | SVAMP | MultiArith | Latent step | ||||
|---|---|---|---|---|---|---|---|
| Method | Full-K | Ours | Gain | Full-K | Ours | Gain | Full-K |
| GPT-2 | |||||||
| CODI | 42.1 | 43.4 | +1.3 | 41.0 | 43.1 | +2.1 | 94.4 |
| SIM-CoT | 39.2 | 41.5 | +2.3 | 40.0 | 43.5 | +3.5 | 91.7 |
| Coconut | 27.1 | 27.7 | +0.6 | 31.4 | 33.7 | +2.3 | 71.7 |
| SIM-CoT Coconut | 44.7 | 44.7 | +0.0 | 40.6 | 42.2 | +1.6 | 92.8 |
| Llama-1B | |||||||
| CODI | 55.5 | 55.7 | +0.2 | 61.1 | 62.3 | +1.2 | 97.2 |
| SIM-CoT | 55.6 | 55.9 | +0.3 | 61.1 | 62.2 | +1.1 | 97.2 |
Key findings:
- Largest gain: MultiArith with GPT-2/Coconut (71.7% → 78.9%)
- Improvements persist even at high accuracy levels (e.g., Llama-1B/CODI on MultiArith: 97.2% → 98.3%)
- Significant latent step savings (up to 29.0%)
5.2 Constructive Progress Across Latent CoT
- Progress completion: Majority of constructive progress is completed by the third latent step across all six reasoners
- Reasoner differences: CODI variants show earlier completion; Coconut variants retain more progress for later steps
- Table 3 shows higher constructive progress consistently corresponds to higher reasoning accuracy:
- GPT-2/SIM-CoT on SVAMP: 13.2% (Lower) vs. 73.4% (Higher)
- Llama-1B/CODI on GSM8K: 24.9% (Lower) vs. 69.8% (Higher)
5.3 Remaining Progress and Computation Returns
Table 4 shows:
- Higher remaining progress () corresponds to larger accuracy gains from Step 3 to Step 4
- Example: GPT-2/SIM-CoT on GSM8K: accuracy increases from 18.5% to 57.3% under higher , vs. remaining at 35.0% under lower
- Future gain differences grow over subsequent latent steps (e.g., Llama-1B/CODI: only 1.6% difference at Step 4, expanding to 58.4% by Step 5)
- Larger decreases in are associated with larger reductions in subsequent reasoning gain
5.4 Continuation Signals Comparison
Table 5 (matched latent computation): Remaining progress outperforms stability, confidence, and convergence signals in most settings.
Table 6 (transferred adaptive methods):
| Dataset | Backbone | Ours | ALR | DEER | Dynamic K-Head |
|---|---|---|---|---|---|
| GSM8K | GPT-2/CODI | 41.1 | 38.4 | 37.3 | 38.4 |
| GSM8K | GPT-2/SIM-CoT | 39.2 | 35.1 | 34.3 | 34.3 |
| SVAMP | GPT-2/SIM-CoT | 40.2 | 39.9 | 39.6 | 39.8 |
| MultiArith | GPT-2/SIM-CoT | 89.4 | 88.9 | 87.1 | 87.2 |
| MultiArith | Llama-1B/CODI | 97.2 | 96.6 | 96.9 | 96.6 |
Remaining progress yields higher accuracy than all transferred criteria by up to 4.9 percentage points, with the largest gap on GSM8K where continued latent computation still yields gains.
Theoretical and Practical Implications
Theoretical Implications
-
Non-uniform reasoning benefits: The paper formally demonstrates that latent CoT's reasoning value is non-uniform along the trajectory, contradicting the implicit assumption that deeper latent reasoning always helps.
-
Causal estimation framework: Proposition 4.1 establishes that remaining progress can be optimally predicted from the latent trajectory through conditional expectation, providing a principled foundation for online adaptation.
-
Monotone allocation principle: Theorem 4.2 shows that under increasing differences in the value function, trajectories with more remaining progress should receive no less additional computation—a formal justification for progress-based budget allocation.
Practical Implications
-
Inference efficiency: The method reduces latent computation by up to 29.0% while improving accuracy, offering immediate computational savings for deployed latent reasoners.
-
No retraining required: The approach works on frozen LLMs, making it a lightweight, plug-and-play solution compatible with existing latent CoT reasoners.
-
Generalizable framework: The constructive-progress framework applies across diverse latent reasoner architectures (CODI, Coconut, SIM-CoT) and backbones (GPT-2, Llama-1B), suggesting broad applicability.
-
Beyond fixed-horizon thinking: The work challenges the field to reconsider fixed-horizon inference as a design choice rather than a default, opening new directions for adaptive inference strategies.
Conclusion
Main Takeaways
-
Completing the full latent horizon can degrade reasoning performance—gold answers reached earlier can be overwritten by later revision stages.
-
Latent CoT evolves through three distinct functional stages (frontier advancement, candidate organization, answer revision), with reasoning gains concentrated in the frontier stage.
-
Adaptive latent inference based on estimated remaining progress consistently improves reasoning accuracy (up to +7.2 percentage points) while reducing latent computation (up to 29.0%) across CODI, Coconut, and SIM-CoT variants on GSM8K, SVAMP, and MultiArith.
-
Higher realized constructive progress corresponds to higher current reasoning accuracy, while more remaining progress corresponds to larger future gains from continued latent CoT.
Future Directions
- Extending adaptive latent inference to more complex reasoning tasks beyond arithmetic benchmarks
- Investigating whether the three-stage evolution pattern generalizes to other latent reasoning architectures
- Exploring training-time adaptations that could make latent CoT more robust to revision-stage degradation
- Developing richer trajectory representations to improve remaining progress estimation
- Scaling the approach to larger models and more diverse reasoning domains
The paper concludes that "by rethinking the fixed-horizon of latent CoT, we develop a method that adaptively adjusts the executed horizon during inference," consistently improving reasoning accuracy while reducing latent computation.
Related papers
- SWE-growth: Teacher-Guided Adaptive Self-Play for Software Engineering Agents
SWE-Growth uses a frozen teacher to guide self-play curricula, improving software agents to 31.7% on SWE-bench Pro and 17.2% on TerminalBench, surpassing student-only training.
- From Repositories to Training Environments: Scaling Long-Horizon Coding Agents with Environment Evolution Loop
RepoFactory's proxy-driven environment evolution loop yields RepoHorizen-35B-A3B, a model improving coding-agent benchmarks by 22.6 points on NL2Repo and 8.5 points on average.
- Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Looped Transformers perform state-dependent computation, not fixed one-loop-one-step algorithms, and a learned linear layer can steer frozen shared weights via attention routing.