# Thinking Inertia: LLMs Keep Thinking When Told Not To

> No-thinking controls in LLMs are unreliable: visible explicit inference persists under strict suppression, scaling with answer-space openness and trading off accuracy on open-ended tasks.

- **Source:** [arXiv](https://arxiv.org/abs/2610.11765)
- **Published:** 2026-10-14
- **Permalink:** https://picx.dev/p/E0QkaC
- **Whiteboard:** https://picx.dev/p/E0QkaC/image

## Summary

## Summary (Overview)

- This paper introduces a **response-centric framework** for measuring "no-thinking" behavior in LLMs, decomposing outputs into pre-answer text $T$ and final answer $A$, evaluated at three levels: Empty-Thinking Rate (ETR), Question–Pre-answer Relevance (QRel.), and Explicit Inference Rate (EIR).
- The authors identify **"Thinking Inertia"**: visible explicit inference persists even under strict no-thinking controls and becomes progressively more prevalent as the answer space opens (Boolean < multiple-choice < open-ended).
- Across six LLMs (Qwen3 family, DeepSeek-V4-Flash, GPT-5.4, Gemini-3-Flash) and six benchmarks, **explicit no-think controls are unreliable**—even DeepSeek-V4-Flash retains visible explicit inference in 99.9% of open-ended responses under its disabled-thinking setting.
- A **no-thinking–performance trade-off** emerges on open-ended tasks: strict answer-only prompting raises mean ETR from near zero to ~40% while reducing mean accuracy by ~15 percentage points.
- **Answer space matters**: rewriting the same questions across answer spaces shows candidate-visible formats make answer-only responses substantially easier to produce than open-generation formats.

---

## Introduction and Theoretical Foundation

### Background and Motivation

Large Language Models (LLMs) benefit substantially from extended reasoning, from early chain-of-thought prompting [42, 19] to test-time scaling [31, 3]. However, extended reasoning raises inference costs in latency and token consumption, motivating work on shortening or compressing reasoning traces [1, 18, 11, 45]. Beyond efficiency, humans often respond from intuition and deliberate only when needed, motivating **adaptive reasoning** capabilities.

### The Proxy Problem

Current practice identifies "no-thinking" through three surface proxies, each with failure cases:

1. **Prompt-side control**: Running the model in a no-think setting does not guarantee the absence of question-conditioned content.
2. **Visible-format control**: The absence of special `thinking` tokens does not prevent reasoning from appearing as ordinary language.
3. **Length-based control**: Short outputs can still encode compact, task-specific reasoning steps.

> "These proxies conflate instruction control with the suppression of visible reasoning. For example, a generic lead-in such as 'Let me think about this. The answer should be <ans>' carries no task-specific content, yet would be counted as reasoning by surface proxies."

### Theoretical Foundation

The paper formalizes no-thinking as an **observable response behavior**. The response is decomposed as:

$$
\pi \big(S(Q)\big) = (T, A)\tag{1}
$$

where $A$ is the last parseable final answer, $T$ is all readable visible text preceding it, and $S(\cdot)$ is an intervention applied to the question $Q$. Each task sample is written as $Q = (q, c, \mathcal{A})$, where $q$ is the question, $c$ is optional context, and $\mathcal{A}$ is the admissible answer space.

---

## Methodology

### Three-Level Response Evaluation

**Level 1: Empty-Thinking Rate (ETR)** — Strict answer-only compliance:

$$
e_{i} = \mathbf{1}\{|T_{i}| = 0\}, \qquad \mathrm{ETR} = 100\% \times \frac{1}{N} \sum_{i=1}^{N} e_{i}\tag{2}
$$

**Level 2: Question–Pre-answer Relevance (QRel.)** — Continuous semantic similarity using instruction-aware Qwen3-Embedding-4B:

$$
\operatorname{QRel.}(Q_{i}, T_{i}) = \begin{cases} 0, & |T_{i}| = 0, \\ \cos\bigl(\phi(Q_{i}), \phi(T_{i})\bigr), & |T_{i}| > 0. \end{cases}\tag{3}
$$

**Level 3: Explicit Inference Rate (EIR)** — Blinded LLM judge (GPT-5.5) classifies non-empty $T$ into five mutually exclusive labels (Table 1): Generic/off-topic, Paraphrase-only, Relevant non-inferential, Explicit reasoning, and Unclear:

$$
\mathrm{EIR} = 100\% \times \frac{1}{N} \sum_{i=1}^{N} \operatorname{Judge}(Q_{i}, T_{i})\tag{4}
$$

### Intervention Spectrum

Six modes organized along a no-think intensity axis:

| Mode | Purpose | enable_thinking | Intervention |
|------|---------|-----------------|--------------|
| M1 THINK-ON | Native thinking-on (−) | ✓ | None |
| M2 THINK-OFF | Native no-thinking (0) | ✕ | None |
| M3 CoT | Explicit reasoning (−) | ✕ | "Think step by step." |
| M4 SOFT | Soft regularization (+) | ✕ | "A very short lead-in is allowed, but do not explain." |
| M5 STRICT | Strict regularization (+++) | ✕ | "Give the final answer directly without any reasoning." |
| M6 PREFIX | Prefix forcing (+++) | ✕ | "The answer is \boxed{" |

### Experimental Setup

- **Models**: Qwen3-4B, Qwen3-32B, Qwen3-235B-A22B, DeepSeek-V4-Flash (284B), GPT-5.4, Gemini-3-Flash
- **Benchmarks**: BoolQ, StrategyQA (Boolean); MMLU, MMLU-Pro (multiple-choice); GSM8K, MATH (open-ended)
- **Human validation**: 600 outputs sampled, 5 annotators, blinded to model identity and intervention

---

## Empirical Validation / Results

### Q1: Native Think-on vs. Think-off

- **Boolean tasks**: Most models reach ~80–100% ETR under THINK-OFF, with sharply reduced QRel. and EIR.
- **MCQ tasks**: Intermediate but heterogeneous—ETR ranges from near zero to 95% across models.
- **Open-ended tasks**: ETR collapses to near zero everywhere; QRel. remains close to THINK-ON levels and EIR remains high.

> "Visible explicit inference is organized more by the task's answer space than by the switch itself."

### Q2: No-Think Variants Performance

- **Prefix forcing (M6) is not a stronger STRICT**: Mean ETR drops from M5 to M6 on MCQ and Open-ended tasks, while QRel. and EIR rebound.
- **Open-ended trade-off**: Moving from M2 to M5 raises mean ETR from near zero to ~40% and reduces mean accuracy by ~15 percentage points. EIR decreases from 99.69% to 62.00% across the complete corpus.
- **Bool and MCQ show little accuracy trade-off**: EIR decreases from 5.15% to 0.12% (Bool) and 45.57% to 25.39% (MCQ), while accuracy remains largely flat.

### Q3: Domain and Answer-Space Variation

**Domain analysis (Table 4)** — Under fixed four-choice MCQ interface:

| MMLU area | Qwen3-4B EIR | Qwen3-32B EIR | Qwen3-235B EIR | DeepSeek EIR |
|-----------|-------------|--------------|---------------|-------------|
| Humanities | 56.6% | 4.1% | 49.8% | 82.7% |
| Social Sciences | 16.2% | 10.7% | 49.6% | 69.2% |
| STEM | 60.2% | 59.7% | 85.5% | 83.5% |

STEM combines the **highest accuracy with the lowest ETR** (0.3–36.4%), arguing against a simple positive relationship between task success and answer-only compliance.

**Answer-space rewrites (Figure 6)** — Same MMLU questions rewritten for Qwen3-32B under M2:

- Boolean rewrite: ETR rises from 10.5% → 62.5%, EIR drops from 86.95% → 36.90%, accuracy nearly unchanged (77.8% → 78.9%)
- Open-ended rewrite: ETR drops to 0%, EIR rises to 90.14%, accuracy drops to 52.6%

### Human Validation

- **ETR boundary audit**: 94.2% agreement (565/600 outputs)
- **QRel. correlation**: Mean Spearman ρ = 0.829 with human relevance judgments
- **EIR agreement**: 98.8% mean binary agreement with GPT-5.5 judge; 96.7% with Claude Opus 4.8
- Human annotations reproduce both the answer-space staircase and persistent visible inference on open-ended tasks under M5

| Evaluator | M2 Bool | M2 MCQ | M2 Open | M5 Bool | M5 MCQ | M5 Open |
|-----------|---------|--------|---------|---------|--------|---------|
| GPT-5.5 | 4.0% | 29.6% | 100.0% | 0.0% | 11.8% | 57.1% |
| Claude Opus 4.8 | 0.0% | 29.6% | 100.0% | 0.0% | 11.8% | 53.6% |
| Human mean | 4.0% | 30.4% | 100.0% | 0.0% | 11.8% | 57.1% |

---

## Theoretical and Practical Implications

### Theoretical Contributions

1. **No-thinking as a measurable behavior**: The paper establishes no-thinking as a distinct, testable capability requiring systematic evaluation alongside reasoning ability itself.
2. **Thinking inertia**: Visible inference persistence is organized by answer space structure, not by model settings or instructions alone.
3. **Answer-space dependence**: The structural support of the question (candidate visibility) governs compressibility, not question difficulty or model capability alone.

### Practical Implications

- **Task-aware routing**: Models should answer directly when the interface provides sufficient support (e.g., Boolean verification) and preserve explicit computation when suppression would sacrifice accuracy (e.g., open-ended generation).
- **Caution on cost assumptions**: Visible compliance with no-thinking instructions should not be equated with lower inference cost when hidden reasoning may remain active.
- **High-stakes settings**: Concise outputs may reduce cost and disclosure, but explanations remain important for verification and error detection.

---

## Conclusion

The paper establishes that "no-thinking" in LLMs should be evaluated as a **response-level property** rather than inferred from surface proxies. The key findings are:

1. **Explicit no-think controls are not reliable**: Even frontier models retain visible explicit inference under disabled-thinking settings.
2. **Thinking inertia**: Visible inference persists and scales with answer-space openness.
3. **No-thinking–performance trade-off**: Stronger suppression improves compliance but can reduce accuracy on open-ended tasks.
4. **Answer space matters**: Candidate-visible formats make answer-only responses easier to produce.

### Future Directions

- **Calibrated stopping**: Train models to jointly optimize accuracy, answer-only compliance, visible relevance/inference, latency, and actual reasoning-token cost.
- **Cross-domain extension**: Test generality across languages, code, tool-use, multi-turn, and stochastic settings.
- **Internal-state analysis**: Examine whether computation disappears, moves to a hidden channel, or is only omitted from view.
- **Auditable decisions**: Make models' decisions about when and how to expose intermediate computation controllable and auditable, rather than suppressing computation universally.

> "Building LLMs that can stop thinking on demand requires more than hiding reasoning traces or enforcing shorter outputs: it requires evaluating when direct answering is actually supported by the task."

---

_Markdown view of https://picx.dev/p/E0QkaC, served by PicX — AI-generated visual whiteboard summaries of research papers._