Summary (Overview)

• Key contribution: This paper demonstrates that Mixture-of-Experts (MoE) router states provide a natural, training-free interface for both reading and controlling reasoning behavior in large language models.
• Behavior readout: Linear probes on router logits predict six reasoning behaviors (planning, execution, restatement, fact, reflection, conclusion) nearly as well as probes on the full residual stream (macro-F1 0.659 vs. 0.658), while using about half as many features.
• Cross-domain generalization: Behavior–expert associations transfer across six datasets spanning five domains (math, code, science, knowledge, commonsense), with mean source-to-target AUC of 0.768.
• Test-time intervention: Selectively editing behavior-associated expert router scores reduces generated tokens by a median of 30.4% (up to 63.3%) across four models and four benchmarks, with accuracy dropping by at most 0.5 percentage points.
• Mechanistic insight: Suppressing planning-associated experts works by redirecting the model's next plan from opening a new round of verification toward wrapping up, rather than by silencing planning altogether.

---

Introduction and Theoretical Foundation

Background and Motivation

Reasoning models allocate test-time compute across distinct behaviors such as planning, deriving, and verifying. However, models frequently "overthink"—continuing to verify and re-derive long after finding a correct answer—which wastes expensive compute and can degrade performance. Existing interventions (token budgets, prompting, steering) primarily regulate reasoning duration rather than targeting specific behaviors.

The paper asks three progressively demanding questions:

1. Can routing read out reasoning behavior? — Do router states expose what the model is currently doing?
2. Do behavior–expert associations persist across domains? — Or are they dataset-specific artifacts?
3. Can they guide useful test-time interventions? — Can we curb overthinking without harming accuracy?

Theoretical Foundation: Sparse MoE Routing

An MoE layer replaces the dense feed-forward sublayer with $n_r$ routed experts and, in some models, $n_s$ always-active shared experts. For token representation $h_t^{(\ell)}$, the router produces logits and selects the k highest-probability routed experts:

$$p_t^{(\ell)} = \mathrm{softmax}(z_t^{(\ell)}), \qquad S_t^{(\ell)} = \mathrm{Top-}K(p_t^{(\ell)}, k), \qquad |S_t^{(\ell)}| = k \ll n_r, \tag{1}$$

The layer output is:

$$\mathrm{MoE}^{(\ell)}(h_t^{(\ell)}) = \sum_{e \in S_t^{(\ell)}} g_{t,e}^{(\ell)} E_e^{(\ell)}(h_t^{(\ell)}) + \sum_{j=1}^{n_s} E_j^{(s,\ell)}(h_t^{(\ell)}). \tag{2}$$

The token–layer sequence of $(z_t^{(\ell)}, S_t^{(\ell)}, g_t^{(\ell)})$ constitutes the routing trace. Expert identities are layer- and model-specific; shared experts (the second term of Eq. 2) are excluded from routing analysis.

Reasoning Behaviors

Six behaviors characterize reasoning spans by their functional role:

Labels are assigned by an LLM judge (GPT-5.5) using the full trace as context, and describe the role of written reasoning rather than whether claims are correct.

---

Methodology

Routing Representations

Two complementary views of routing are used:

• For behavior readout: Continuous pre-top-k logits $z_t^{(\ell)}$ retain scores for every routed expert, including unselected ones.
• For expert identification: Post-top-k selection indicators measure participation in executed computation:

$$r_{t,e}^{(\ell)} = \begin{cases} 1, & e \in S_t^{(\ell)}, \\ 0, & \text{otherwise}. \end{cases} \tag{3}$$

Behavior–Expert Association

At each layer, experts are ranked by the absolute increase in selection probability for behavior $b$ relative to dataset $d$ overall (risk difference):

$$a_{b,d,e}^{(\ell)} = p(e \text{ selected } | b, d, \ell) - p(e \text{ selected } | d, \ell). \tag{4}$$

This controls for differences in baseline expert usage across datasets.

Steering Operator

Steering modifies router scores at inference time without updating parameters. Raw logits are mapped to log-probabilities:

$$s_t^{(\ell)} = \log \mathrm{softmax}(z_t^{(\ell)}), \tag{5}$$

With activation set $A_+^{(m,\ell)}$ and deactivation set $A_-^{(m,\ell)}$, and margin $\epsilon > 0$:

$$\widehat{s}_{t,e}^{(\ell)} = \begin{cases} s_{t,\max}^{(\ell)} + \epsilon, & e \in A_+^{(m,\ell)}, \\ s_{t,\min}^{(\ell)} - \epsilon, & e \in A_-^{(m,\ell)}, \\ s_{t,e}^{(\ell)}, & \text{otherwise.} \end{cases} \tag{6}$$

The model then recomputes its standard softmax, top-k set, and gate normalization.

Experimental Setup

• Probing: Qwen3-30B-A3B, 330 traces on GSM8K, NuminaMath, and AIME 2023 (78,955 labeled segments), L2-regularized multinomial logistic probe.
• Expert identification: 1,980 traces across six datasets (math mixture, MATH500, LiveCodeBench, GPQA, MMLU-Pro, CommonsenseQA).
• Steering: Four models (Gemma-4, two Qwen3 generations, GPT-OSS) and four benchmarks (MATH500, AIME 2026, GPQA-Diamond, AICrypto), with development-split selection of expert sets.

Baselines: (1) unmodified model, (2) equally sized random expert sets with same intervention direction, (3) brevity prompt via system instruction.

---

Empirical Validation / Results

Behavior Readout from Routing (Section 4)

• Single-token readout: Dev-selected layer L12 reaches 0.608 macro-F1 (95% CI [0.584, 0.630]), far above the 0.096 majority-class baseline.
• Router vs. residual stream: Best residual layer (L14) reaches 0.658. Concatenating router logits from a nine-layer window (L8–L16) reaches 0.659, within 0.1 points of the residual stream at 56% of its width.
• Not a surface shortcut: Router features (0.709) beat input embeddings (0.624), lexical TF-IDF (0.566), and trace position (0.124). Dropping the first five tokens of each segment still yields 0.677, above the 0.626 from those tokens alone.

Cross-Domain Transfer (Section 5)

• Layer 12 shows same-behavior correlation 0.729 vs. different-behavior correlation −0.107.
• Top-10 expert sets have mean cross-dataset Jaccard 0.511 (vs. 0.202 null), with 15–23% of routing mass.
• Source-to-target transfer AUC at layer 36: 0.768 overall; reflection (0.863) and conclusion (0.811) transfer best, execution weakest (0.631).

Steering Results (Section 6)

Token reduction and accuracy:

Baseline comparisons:

• Random expert interventions increase token generation in 14/16 settings.
• Routing interventions save more tokens than the brevity prompt in all 16 settings.

Why Suppressing Planning Works (Section 6.4)

On 100 MATH500 problems with Qwen3-30B-A3B:

• Suppressing 200 planning experts: −22% tokens
• Suppressing reflection experts: +5% tokens
• Suppressing restatement experts: +1% tokens (more than doubles post-answer checking)
• Suppressing execution experts: +52% tokens (or drops accuracy to 40% when applied strongly)

Key mechanism: Unmodified models reach their first answer early (median 8–21% of the trace depending on model), and 82–88% of planning sentences occur after the first answer. Suppressing planning:

1. Leaves pre-answer reasoning essentially untouched (422 vs. 441 tokens to first answer)
2. Reduces post-answer verification rounds
3. Redirects the next plan from opening a new angle ("Alternative thought:") toward wrapping up (68 wrap-ups vs. 11 new angles, vs. 23/23 unmodified)
4. Increases likelihood of ending thinking within ~170 tokens (17% vs. 5%)

---

Theoretical and Practical Implications

Theoretical Significance

1. Routing as an interpretable lens: The paper establishes that MoE routing states encode functional reasoning roles, not just lexical or topical features. This provides a new window into the internal computation of reasoning models.

2. Behavioral decomposition of overthinking: The finding that overthinking is driven by planning (which launches new verification rounds) rather than reflection (which merely voices doubts) challenges the intuitive assumption that overthinking is primarily an excess of verification. This reframes how we understand the structure of redundant computation.

3. Expert specialization is behaviorally organized: Behavior–expert associations transfer across domains, suggesting that MoE models develop stable, domain-general computational modules for reasoning functions.

Practical Implications

1. Training-free efficiency optimization: The steering operator provides a lightweight, inference-time mechanism for reducing compute waste without fine-tuning or prompting, which is particularly valuable for deployed reasoning models.

2. Targeted control beyond duration: Unlike token budgets that regulate how much reasoning, this approach controls what kind of reasoning, enabling more surgical interventions.

3. Model-agnostic applicability: The method works across four different MoE architectures with varying expert pool sizes, routing sparsity, and shared-expert configurations, suggesting broad applicability.

4. Limitations: The paper notes that behavior associations alone do not establish what an expert causally controls, and future work should test whether the same experts govern reasoning in longer-horizon and agentic tasks.

---

Conclusion

This paper demonstrates that MoE routing provides both an interpretable signal of a model's reasoning state and a training-free mechanism for steering how much computation it spends. The key findings are:

1. Router states reveal reasoning behavior — Linear probes on router logits recover six reasoning behaviors about as accurately as residual-stream probes while using half as many features.

2. Behavior–expert associations generalize across domains — Expert sets selected on one dataset predict behavior on held-out domains, indicating stable behavioral organization in MoE models.

3. Targeted steering curbs overthinking — Suppressing planning-associated experts reduces generated tokens by a median of 30.4% (up to 63.3%) across four models and four benchmarks, with accuracy dropping by at most 0.5 points.

4. The mechanism is redirection, not suppression — Steering works not by preventing planning but by redirecting the model's next plan from opening a new verification round toward wrapping up, effectively teaching the model to stop once the problem is solved.

Future directions include testing whether the same experts govern reasoning in longer-horizon and agentic tasks, and exploring whether behavior-associated experts can inform more fine-grained control over reasoning quality beyond length.