When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
Authors: Youngsik Yoon, Siwei Wang, Wei Chen, Jungseul Ok (POSTECH & Microsoft Research Asia)
Summary (Overview)
- Core finding: The standard top-k router in MoE language models is well-aligned with route utility on confident tokens but systematically misaligned on "fragile" tokens—the low-confidence tokens that drive hard reasoning—where lower-loss equal-compute routes consistently exist inside the frozen model but are not selected.
- Token-conditional misalignment: On Fragile tokens, the standard route is best among alternatives only ~0.8% of the time, while the best sampled equal-compute route improves next-token probability by up to 20.4 percentage points.
- Structural cause identified: The authors formalize a "counterfactual blind spot" in standard MoE training—the language modeling loss evaluates only the executed route, and load-balancing losses depend only on aggregate routing statistics, providing no token-level signal on unexecuted equal-compute alternatives.
- Empirical validation: The pattern holds across four open-weight MoE models (Qwen3-30B-A3B, GPT-OSS-20B, DeepSeek-V2-Lite, OLMoE-1B-7B), across layers, and across multiple benchmarks (MATH Level-5, AIME, HMMT, GPQA-Diamond).
- Router-only intervention: A minimal update to only the final-layer router (Expert Preference Optimization, EPO, <0.001% of parameters) shifts pass@K on AIME 2024+2025 and HMMT 2025, demonstrating that part of the failure is router-reachable misallocation, not purely expert capacity limits.
Introduction and Theoretical Foundation
Background
Mixture-of-Experts (MoE) language models scale model capacity by activating only a small subset of parameters per token. The router—typically a single learned linear projection followed by top-k selection—decides which experts process each token. While attention and expert components receive extensive scrutiny, the router has been treated largely as an implementation detail of sparsity, with load-balancing regularizers and downstream performance taken as sufficient evidence of good routing.
Key Question
The paper asks a fundamental but previously unaddressed question: Are the routes selected by a trained top-k router actually good ones? Rather than evaluating routing through aggregate statistics or downstream benchmarks, the authors perform a counterfactual analysis: holding the model fixed, they compare each standard route against sampled equal-compute alternatives for the same token, scoring each by the next-token probability it assigns to the realized token in a verified reasoning trajectory.
Theoretical Foundation
The paper formalizes a counterfactual blind spot in standard MoE training. For a token , only the executed route enters the forward pass. The gradient of the cross-entropy loss with respect to router score within a fixed top-k cell is:
The indicator term is crucial: for experts not in the executed route, the gradient vanishes—their outputs never enter the computation graph. Two distinct routes with produce different cross-entropies and on the same token, but standard training observes only one of them.
Load Balancing Is Aggregate, Not Token-Level
The authors formally show that load-balancing losses do not provide token-level counterfactual signal. For the Switch-style loss with batch of tokens:
the gradient is:
This depends only on aggregate statistics and , not on expert outputs or counterfactual cross-entropies . The same property holds for GShard, Importance/Load losses, and auxiliary-loss-free bias updates.
Methodology
Analysis Protocol
- Models: Four open-weight MoE models are analyzed:
| Model | Total / Active Params | Layers | N / k |
|---|---|---|---|
| Qwen3-30B-A3B | 30.5B / 3.3B | 48 | 128 / 8 |
| GPT-OSS-20B | 21B / 3.6B | 24 | 32 / 4 |
| DeepSeek-V2-Lite | 16B / 2.4B | 27 | 64 / 6 + 2 |
| OLMoE-1B-7B | 7B / 1.3B | 16 | 64 / 8 |
-
Trajectories: Verified-correct reasoning trajectories are generated per problem (MATH Level-5, AIME, HMMT, GPQA-Diamond), restricted to assistant-response tokens.
-
Alternative route sampling: For token at layer , a candidate pool of the highest-scoring experts is formed. Then equal-compute alternatives are drawn via Gumbel-top-k sampling: with .
-
Intervention: For each candidate route , the standard route is replaced at position , gate weights are recomputed, and the forward pass runs with all other parameters fixed.
-
Metrics:
- : next-token probability assigned to the realized token under route
- Top-K rates: fraction of tokens where the standard route ranks within top K of the 33 candidates
- Confidence stratification by route-averaged probability into Confident (), Ambiguous (–), and Fragile ()
Expert Preference Optimization (EPO)
For the router-only intervention, EPO updates only the final-layer router:
- Hard token filter: Tokens with next-token cross-entropy exceeding under the trainable router
- Preference construction: For each hard token, sample alternative routes; if the lowest-CE sample improves over the current route, use it as the chosen route (rejected route is )
- Objective (CE-gap-weighted DPO-style):
where is the observed CE-gap weight. Since and have the same cardinality, shared experts cancel, and the update acts only on distinguishing experts.
Empirical Validation / Results
Core Pattern (Qwen3-30B-A3B, final MoE layer, MATH Level-5)
| Confidence | Tokens (%) | Top-1 (%) | (%) | (%) | Gap (pp) |
|---|---|---|---|---|---|
| Confident () | 78.7 | 51.9 | 99.6 | 99.8 | 0.2 |
| Ambiguous () | 14.3 | 1.5 | 77.9 | 88.9 | 11.0 |
| Fragile () | 6.9 | 0.8 | 40.2 | 60.6 | 20.4 |
Key observations:
- On Confident tokens, all routes perform nearly perfectly—routing choice has little consequence
- On Fragile tokens, the standard route is best only 0.8% of the time, yet the best alternative improves probability by 20.4 pp
- Top-5 and Top-10 rates also collapse outside the Confident range
Consistency Across Layers (Qwen3-30B-A3B)
| Layer | Fragile Top-1 (%) | Fragile Gap (pp) |
|---|---|---|
| L0 (early) | 4.1 | 11.4 |
| L24 (middle) | 2.9 | 12.8 |
| L47 (final) | 0.8 | 20.4 |
The Fragile-token gap grows monotonically with depth—consistent with errors at the final layer feeding directly into the next-token distribution with no further routing to correct them.
Consistency Across Domains and Models
- Benchmarks: AIME (Fragile gap: 19.4 pp), HMMT (19.0 pp), GPQA-Diamond (21.2 pp)—same qualitative pattern
- Models: GPT-OSS-20B (Fragile gap: 11.8 pp), DeepSeek-V2-Lite (24.9 pp), OLMoE-1B-7B (20.8 pp)—pattern holds across all, with varying absolute levels
Router-Only Update Results (EPO)
Pass@K on AIME 2024+2025 and HMMT 2025 shifts above the standard curve for both Qwen3-30B-A3B and GPT-OSS-20B, with 95% bootstrap bands separating over part of the reported range. Table 5 decomposes the effect:
| Confident (A→B→C) | Ambiguous (A→B→C) | Fragile (A→B→C) | |
|---|---|---|---|
| Tokens (%) | 73.2→74.5→75.8 | 17.0→16.0→16.0 | 9.8→9.6→8.2 |
| Top-1 (%) | 42.7→27.0→26.3 | 1.2→2.4→1.9 | 0.6→1.9→2.4 |
| Top-5 (%) | 63.4→43.5→43.3 | 9.4→16.8→16.2 | 11.2→17.9→20.7 |
| Gap (pp) | 0.3→0.3→0.3 | 10.8→11.5→11.3 | 19.2→21.9→20.5 |
Interpretation:
- EPO shifts token distribution toward easier bins (Confident fraction increases, Fragile decreases)
- Route-rank gains occur mainly outside the Confident bin
- The Confident-bin rank drop is not a meaningful probability loss (, gap stays at 0.3 pp)—EPO reduces easy-token-biased rank alignment while improving alignment where route choice matters
Theoretical and Practical Implications
Theoretical Significance
-
First counterfactual evaluation of routing quality: The paper provides the first systematic framework for asking whether trained routers select good routes, independent of downstream performance.
-
Formal characterization of the blind spot: Equation (3) rigorously demonstrates why standard MoE training cannot optimize routing decisions beyond the executed route—the gradient vanishes for unselected experts.
-
Separation of capacity vs. routing failure: The EPO experiment shows that high loss on hard tokens is partially attributable to router-reachable misallocation rather than pure expert capacity limits.
-
Token-conditional nature of routing quality: The finding that routing quality degrades precisely on the tokens where it matters most (fragile, hard-reasoning tokens) suggests that aggregate routing metrics systematically hide the most consequential failures.
Practical Implications
-
Routing as a first-class training target: Routing quality on hard tokens warrants explicit attention in MoE training objectives, not just load-balancing and stability regularizers.
-
Potential for counterfactual-aware pretraining: The authors suggest designing pretraining objectives that place token-level signal on equal-compute alternative routes—e.g., lightweight evaluation of sampled alternatives alongside the executed route, incorporating their cross-entropies into the loss.
-
Router-only adaptation as a lightweight intervention: EPO demonstrates that meaningful shifts in reasoning performance can be achieved by updating <0.001% of parameters, suggesting cost-effective post-training routing improvements.
-
Diagnostic value: The confidence-stratified routing analysis provides a new diagnostic tool for evaluating MoE model quality beyond aggregate metrics.
Conclusion
Main Takeaways
- The standard top-k router in MoE language models is well-aligned with route utility on confident tokens but uninformative on fragile tokens that drive hard reasoning.
- This pattern is structurally inevitable given how standard MoE training evaluates routing: the LM loss scores only the executed route, and load balancing depends only on aggregate statistics—neither provides token-level counterfactual signal.
- A minimal router-only update (EPO, <0.001% of parameters) shifts pass@K on mathematical reasoning benchmarks, proving that part of the failure reflects router-reachable misallocation rather than expert capacity limits.
- The pattern is consistent across four MoE families, layers, and benchmarks, pointing to a shared cause independent of specific router training.
Future Directions
- Counterfactual-aware pretraining objectives that evaluate sampled alternative routes alongside the executed one
- Joint multi-layer routing interventions (the current analysis holds other layers fixed; whether per-layer failures compound or cancel is open)
- Refined proxies for routing quality beyond verified-correct trajectory probability (the current proxy may overstate improvement room on tokens where the realized token was a fortunate sample)
- Scaling counterfactual routing evaluation to pretraining scale without sacrificing sparse computation efficiency
Limitations
- Each layer's routing is analyzed independently; joint multi-layer modification is not addressed
- Analysis is restricted to verified-correct trajectories, where the realized next token already lies on a successful path
- The probability proxy may overstate the room for improvement on tokens where the realized token was itself a fortunate sample
Related papers
- RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding
RIPPLE internalizes prefix-only recovery guidance into a single policy via stage-specific credit, improving coding agents by 4.8 points across four benchmarks without deployment-time critics.
- UpLoop: Training Looped Transformers via Upcycling at No Extra Pretraining Compute
UpLoop trains looped Transformers with identical parameters, tokens, and FLOPs to dense baselines, improving reasoning and language modeling across scales via progressive depth growth and shared-weight looping.
- Rethinking Fixed-Horizon Latent Reasoning
Adaptive latent inference that stops reasoning when progress stalls improves accuracy by up to 7.2 points while cutting computation by 29%, as completing fixed latent horizons can degrade performance.