# When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models

> Standard top-k MoE routers are systematically misaligned on the low-confidence tokens driving hard reasoning, leaving better equal-compute routes unused, yet router-only updates can recover them.

- **Source:** [arXiv](https://arxiv.org/abs/2605.07260)
- **Published:** 2026-08-17
- **Permalink:** https://picx.dev/p/fbbSgs
- **Whiteboard:** https://picx.dev/p/fbbSgs/image

## Summary

# When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models

**Authors:** Youngsik Yoon, Siwei Wang, Wei Chen, Jungseul Ok (POSTECH & Microsoft Research Asia)

---

## Summary (Overview)

- **Core finding:** The standard top-k router in MoE language models is well-aligned with route utility on confident tokens but systematically misaligned on "fragile" tokens—the low-confidence tokens that drive hard reasoning—where lower-loss equal-compute routes consistently exist inside the frozen model but are not selected.
- **Token-conditional misalignment:** On Fragile tokens, the standard route is best among alternatives only ~0.8% of the time, while the best sampled equal-compute route improves next-token probability by up to 20.4 percentage points.
- **Structural cause identified:** The authors formalize a "counterfactual blind spot" in standard MoE training—the language modeling loss evaluates only the executed route, and load-balancing losses depend only on aggregate routing statistics, providing no token-level signal on unexecuted equal-compute alternatives.
- **Empirical validation:** The pattern holds across four open-weight MoE models (Qwen3-30B-A3B, GPT-OSS-20B, DeepSeek-V2-Lite, OLMoE-1B-7B), across layers, and across multiple benchmarks (MATH Level-5, AIME, HMMT, GPQA-Diamond).
- **Router-only intervention:** A minimal update to only the final-layer router (Expert Preference Optimization, EPO, <0.001% of parameters) shifts pass@K on AIME 2024+2025 and HMMT 2025, demonstrating that part of the failure is router-reachable misallocation, not purely expert capacity limits.

---

## Introduction and Theoretical Foundation

### Background

Mixture-of-Experts (MoE) language models scale model capacity by activating only a small subset of parameters per token. The **router**—typically a single learned linear projection followed by top-k selection—decides which experts process each token. While attention and expert components receive extensive scrutiny, the router has been treated largely as an implementation detail of sparsity, with load-balancing regularizers and downstream performance taken as sufficient evidence of good routing.

### Key Question

The paper asks a fundamental but previously unaddressed question: **Are the routes selected by a trained top-k router actually good ones?** Rather than evaluating routing through aggregate statistics or downstream benchmarks, the authors perform a counterfactual analysis: holding the model fixed, they compare each standard route against sampled equal-compute alternatives for the same token, scoring each by the next-token probability it assigns to the realized token in a verified reasoning trajectory.

### Theoretical Foundation

The paper formalizes a **counterfactual blind spot** in standard MoE training. For a token $t$, only the executed route $S_t^{\text{std}}$ enters the forward pass. The gradient of the cross-entropy loss $\mathcal{L}_t$ with respect to router score $s_{tj}$ within a fixed top-k cell is:

$$
\frac{\partial \mathcal{L}_t}{\partial s_{tj}} = \mathbf{1}\{j \in S_t^{\text{std}}\} \, g_{tj}^{\text{std}} \Big(\nabla_{h_t^{\text{std}}} \mathcal{L}_t \cdot (E_j(x_t) - h_t^{\text{std}})\Big).\tag{3}
$$

The indicator term is crucial: for experts **not** in the executed route, the gradient vanishes—their outputs never enter the computation graph. Two distinct routes $S, S' \subseteq \{1, \dots, N\}$ with $|S| = |S'| = k$ produce different cross-entropies $\mathcal{L}_t(S)$ and $\mathcal{L}_t(S')$ on the same token, but standard training observes only one of them.

### Load Balancing Is Aggregate, Not Token-Level

The authors formally show that load-balancing losses do **not** provide token-level counterfactual signal. For the Switch-style loss with batch of $T$ tokens:

$$
f_i = \frac{1}{Tk} \sum_{t=1}^{T} \mathbf{1}\{i \in S_t^{\text{std}}\}, \qquad \bar{p}_i = \frac{1}{T} \sum_{t=1}^{T} p_{ti},\tag{4}
$$

the gradient is:

$$
\nabla_{s_t} \mathcal{R}_{\text{lb}} = \frac{\lambda N}{T} \big(\text{diag}(p_t) - p_t p_t^\top\big) f.\tag{5}
$$

This depends only on aggregate statistics $f$ and $p_t$, not on expert outputs $\{E_j(x_t)\}$ or counterfactual cross-entropies $\{\mathcal{L}_t(S)\}$. The same property holds for GShard, Importance/Load losses, and auxiliary-loss-free bias updates.

---

## Methodology

### Analysis Protocol

1. **Models:** Four open-weight MoE models are analyzed:

| Model | Total / Active Params | Layers | N / k |
|-------|----------------------|--------|-------|
| Qwen3-30B-A3B | 30.5B / 3.3B | 48 | 128 / 8 |
| GPT-OSS-20B | 21B / 3.6B | 24 | 32 / 4 |
| DeepSeek-V2-Lite | 16B / 2.4B | 27 | 64 / 6 + 2 |
| OLMoE-1B-7B | 7B / 1.3B | 16 | 64 / 8 |

2. **Trajectories:** Verified-correct reasoning trajectories are generated per problem (MATH Level-5, AIME, HMMT, GPQA-Diamond), restricted to assistant-response tokens.

3. **Alternative route sampling:** For token $t$ at layer $\ell$, a candidate pool $P_{t,\ell}$ of the $m=32$ highest-scoring experts is formed. Then $G=32$ equal-compute alternatives are drawn via Gumbel-top-k sampling: $S_{t,\ell}^{(g)} = \text{TopK}_{i \in P_{t,\ell}}(s_{t,\ell,i} + \epsilon^{(g)}, k)$ with $\epsilon^{(g)} \sim \text{Gumbel}(0,1)^m$.

4. **Intervention:** For each candidate route $S$, the standard route is replaced at position $(t,\ell)$, gate weights are recomputed, and the forward pass runs with all other parameters fixed.

5. **Metrics:**
   - $p_{t,\ell}(S)$: next-token probability assigned to the realized token under route $S$
   - **Top-K rates**: fraction of tokens where the standard route ranks within top K of the 33 candidates
   - **Confidence stratification** by route-averaged probability $\bar{p}_{t,\ell}$ into Confident ($>0.9$), Ambiguous ($0.5$–$0.9$), and Fragile ($\leq 0.5$)

### Expert Preference Optimization (EPO)

For the router-only intervention, EPO updates only the final-layer router:

- **Hard token filter:** Tokens with next-token cross-entropy exceeding $\tau = 0.1$ under the trainable router
- **Preference construction:** For each hard token, sample $G$ alternative routes; if the lowest-CE sample improves over the current route, use it as the chosen route $r_t^+$ (rejected route is $r_t^-$)
- **Objective** (CE-gap-weighted DPO-style):

$$
\ell_t = -\Delta_t \log \sigma \left(\beta \log \frac{\pi_\theta(r_t^+ \mid x_t)}{\pi_{\text{ref}}(r_t^+ \mid x_t)} - \beta \log \frac{\pi_\theta(r_t^- \mid x_t)}{\pi_{\text{ref}}(r_t^- \mid x_t)}\right),\tag{6}
$$

where $\Delta_t = \max(0, \mathcal{L}(r_t^-) - \mathcal{L}(r_t^+))$ is the observed CE-gap weight. Since $r_t^+$ and $r_t^-$ have the same cardinality, shared experts cancel, and the update acts only on distinguishing experts.

---

## Empirical Validation / Results

### Core Pattern (Qwen3-30B-A3B, final MoE layer, MATH Level-5)

| Confidence | Tokens (%) | Top-1 (%) | $p_{\text{std}}$ (%) | $p_{\text{best}}$ (%) | Gap (pp) |
|------------|-----------|-----------|---------------------|---------------------|----------|
| Confident ($\bar{p} > 0.9$) | 78.7 | 51.9 | 99.6 | 99.8 | 0.2 |
| Ambiguous ($0.5 < \bar{p} \leq 0.9$) | 14.3 | 1.5 | 77.9 | 88.9 | 11.0 |
| Fragile ($\bar{p} \leq 0.5$) | 6.9 | 0.8 | 40.2 | 60.6 | 20.4 |

**Key observations:**
- On Confident tokens, all routes perform nearly perfectly—routing choice has little consequence
- On Fragile tokens, the standard route is best only 0.8% of the time, yet the best alternative improves probability by 20.4 pp
- Top-5 and Top-10 rates also collapse outside the Confident range

### Consistency Across Layers (Qwen3-30B-A3B)

| Layer | Fragile Top-1 (%) | Fragile Gap (pp) |
|-------|------------------|-----------------|
| L0 (early) | 4.1 | 11.4 |
| L24 (middle) | 2.9 | 12.8 |
| L47 (final) | 0.8 | 20.4 |

The Fragile-token gap grows monotonically with depth—consistent with errors at the final layer feeding directly into the next-token distribution with no further routing to correct them.

### Consistency Across Domains and Models

- **Benchmarks:** AIME (Fragile gap: 19.4 pp), HMMT (19.0 pp), GPQA-Diamond (21.2 pp)—same qualitative pattern
- **Models:** GPT-OSS-20B (Fragile gap: 11.8 pp), DeepSeek-V2-Lite (24.9 pp), OLMoE-1B-7B (20.8 pp)—pattern holds across all, with varying absolute levels

### Router-Only Update Results (EPO)

Pass@K on AIME 2024+2025 and HMMT 2025 shifts above the standard curve for both Qwen3-30B-A3B and GPT-OSS-20B, with 95% bootstrap bands separating over part of the reported range. Table 5 decomposes the effect:

| | Confident (A→B→C) | Ambiguous (A→B→C) | Fragile (A→B→C) |
|---|---|---|---|
| Tokens (%) | 73.2→74.5→75.8 | 17.0→16.0→16.0 | 9.8→9.6→8.2 |
| Top-1 (%) | 42.7→27.0→26.3 | 1.2→2.4→1.9 | 0.6→1.9→2.4 |
| Top-5 (%) | 63.4→43.5→43.3 | 9.4→16.8→16.2 | 11.2→17.9→20.7 |
| Gap (pp) | 0.3→0.3→0.3 | 10.8→11.5→11.3 | 19.2→21.9→20.5 |

**Interpretation:**
- EPO shifts token distribution toward easier bins (Confident fraction increases, Fragile decreases)
- Route-rank gains occur mainly outside the Confident bin
- The Confident-bin rank drop is not a meaningful probability loss ($p_{\text{std}} \approx 99.5$, gap stays at 0.3 pp)—EPO reduces easy-token-biased rank alignment while improving alignment where route choice matters

---

## Theoretical and Practical Implications

### Theoretical Significance

1. **First counterfactual evaluation of routing quality:** The paper provides the first systematic framework for asking whether trained routers select good routes, independent of downstream performance.

2. **Formal characterization of the blind spot:** Equation (3) rigorously demonstrates why standard MoE training cannot optimize routing decisions beyond the executed route—the gradient vanishes for unselected experts.

3. **Separation of capacity vs. routing failure:** The EPO experiment shows that high loss on hard tokens is partially attributable to router-reachable misallocation rather than pure expert capacity limits.

4. **Token-conditional nature of routing quality:** The finding that routing quality degrades precisely on the tokens where it matters most (fragile, hard-reasoning tokens) suggests that aggregate routing metrics systematically hide the most consequential failures.

### Practical Implications

1. **Routing as a first-class training target:** Routing quality on hard tokens warrants explicit attention in MoE training objectives, not just load-balancing and stability regularizers.

2. **Potential for counterfactual-aware pretraining:** The authors suggest designing pretraining objectives that place token-level signal on equal-compute alternative routes—e.g., lightweight evaluation of sampled alternatives alongside the executed route, incorporating their cross-entropies into the loss.

3. **Router-only adaptation as a lightweight intervention:** EPO demonstrates that meaningful shifts in reasoning performance can be achieved by updating <0.001% of parameters, suggesting cost-effective post-training routing improvements.

4. **Diagnostic value:** The confidence-stratified routing analysis provides a new diagnostic tool for evaluating MoE model quality beyond aggregate metrics.

---

## Conclusion

### Main Takeaways

1. The standard top-k router in MoE language models is **well-aligned with route utility on confident tokens** but **uninformative on fragile tokens** that drive hard reasoning.
2. This pattern is **structurally inevitable** given how standard MoE training evaluates routing: the LM loss scores only the executed route, and load balancing depends only on aggregate statistics—neither provides token-level counterfactual signal.
3. A **minimal router-only update** (EPO, <0.001% of parameters) shifts pass@K on mathematical reasoning benchmarks, proving that part of the failure reflects router-reachable misallocation rather than expert capacity limits.
4. The pattern is **consistent across four MoE families, layers, and benchmarks**, pointing to a shared cause independent of specific router training.

### Future Directions

- **Counterfactual-aware pretraining objectives** that evaluate sampled alternative routes alongside the executed one
- **Joint multi-layer routing interventions** (the current analysis holds other layers fixed; whether per-layer failures compound or cancel is open)
- **Refined proxies** for routing quality beyond verified-correct trajectory probability (the current proxy may overstate improvement room on tokens where the realized token was a fortunate sample)
- **Scaling counterfactual routing evaluation** to pretraining scale without sacrificing sparse computation efficiency

### Limitations

- Each layer's routing is analyzed independently; joint multi-layer modification is not addressed
- Analysis is restricted to verified-correct trajectories, where the realized next token already lies on a successful path
- The probability proxy may overstate the room for improvement on tokens where the realized token was itself a fortunate sample

---

_Markdown view of https://picx.dev/p/fbbSgs, served by PicX — AI-generated visual whiteboard summaries of research papers._