# From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts

> Routing effective rank from expert-routing traces reveals a reproducible low-high-low trajectory in Mixture-of-Experts reasoning cohorts, with common-mode mass driving two-thirds of the spectral motion.

- **Source:** [arXiv](https://arxiv.org/abs/2609.06403)
- **Published:** 2026-09-12
- **Permalink:** https://picx.dev/p/7fVNsz
- **Whiteboard:** https://picx.dev/p/7fVNsz/image

## Summary

## Summary (Overview)

- The paper introduces **routing effective rank** ($d_{\text{eff}}$), a novel label-free diagnostic that measures the entropy-effective dimensionality of a cross-rollout graph built from Mixture-of-Experts (MoE) expert-routing similarity, providing a spectral lens on how reasoning cohorts reorganize during test-time inference.
- Across **ten MoE configurations**, **five math/science benchmarks**, and **3,105 model–question cohorts**, $d_{\text{eff}}$ exhibits a reproducible **low–high–low trajectory** with a prominent interior maximum in **98.5%** of cohorts: routing similarity is concentrated early, maximally differentiated at intermediate budgets, and reconcentrated later.
- An **exact spectral decomposition** ($\log d_{\text{eff}} = h(m) + (1-m)\log d_{\perp}$) separates cohort-wide common-mode mass from residual spectral dimensionality, revealing that common-mode reallocation accounts for ~**two-thirds** of the trajectory while residual spectrum contributes ~**one-quarter** with substantial independent variation (grouped-CV $R^2 = 0.37$).
- **Behavioral localization** shows: common-mode concentration strongly predicts same-answer recoverability (β = +0.522), and higher reasoning effort delays the peak by **+2.59 octaves** (doublings of token budget) and widens the high-rank regime across all four tested architectures—localizing the effort effect in timing and duration rather than peak amplitude.
- The diagnostic is **practical**: it survives frozen transfer to held-out code generation (97.7% peak rate on LiveCodeBench-v5) and can be computed from sparse expert-activation indicators alone, making it applicable during serving before any answer is graded.

## Introduction and Theoretical Foundation

Test-time scaling transforms inference into a **population process**: instead of one answer, a model produces a cohort of candidate reasoning trajectories combined via majority vote, verifiers, or learned rewards. While most existing analyses characterize cohorts through final answers (votes, margins, verifier scores), the paper argues that **before answers are compared, the cohort already has internal organization**—some trajectories compute similarly, others route through different expert patterns.

**Sparse Mixture-of-Experts (MoE) models** expose this organization through routing traces, which are by-products of decoding (arriving with every token) rather than added probes. Comparing traces across same-question rollouts yields an activation-only similarity graph whose spectrum compactly describes cohort geometry.

The central quantity is **routing effective rank** $d_{\text{eff}}$, the exponential of spectral entropy applied to the routing-similarity spectrum—the same functional as the Vendi score on similarity matrices. Unlike ordinary rank, it discounts numerically tiny directions; unlike graph density, it summarizes the full eigenvalue distribution.

The paper addresses three research questions:
- **RQ1**: Temporal regularity and architecture-dependent timing of the trajectory
- **RQ2**: The spectral channels (common-mode vs. residual) that account for the trajectory
- **RQ3**: How those channels relate to answer organization and reasoning effort

**Key theoretical insight**: The interpretation of $d_{\text{eff}}$ depends on how spectral mass is allocated—a rise may reflect a weaker cohort-wide routing mode, a richer residual spectrum, or both. This motivates studying $d_{\text{eff}}$ as a *decomposable trajectory* rather than an isolated scalar.

## Methodology

### 3.1 Cohorts and the Routing Graph

For each question, a cohort of **$N=64$ rollouts** is sampled and evaluated along a logarithmic budget axis of token prefixes $b \in \{32, 48, \dots, 32768\}$ (16 points), plus a per-rollout commit window at each boxed final answer. At budget $b$, each rollout $i$ has a histogram $h_i(b)$ accumulating the router's gate weight over each (layer, expert) slot in a short trailing window (16 tokens by default).

The symmetric cohort graph $\mathbf{W} \in \mathbb{R}^{N \times N}$ holds **weighted Jaccard similarity**:

$$
\mathbf{W}_{ij}(b) = \frac{\sum_{e} \min\bigl(h_i^{(e)}(b), h_j^{(e)}(b)\bigr)}{\sum_{e} \max\bigl(h_i^{(e)}(b), h_j^{(e)}(b)\bigr)},\tag{1}
$$

indexed over expert slots $e$. Edge weights are computed from router activations only; answer annotations never enter the similarity.

### 3.2 Routing Effective Rank

With $p_i = \lambda_i / \sum_j \lambda_j$ over the eigenvalues of the symmetrized $\mathbf{W}$, the effective rank is:

$$
d_{\text{eff}}(\mathbf{W}) = \exp\Bigl(-\sum_{i} p_i \log p_i\Bigr),\tag{2}
$$

the entropy-effective dimensionality of the routing-similarity spectrum. Under approximately block-structured geometry, it approximates an effective number of balanced routing groups; more generally, it quantifies spectral dimensionality rather than a literal cluster count.

### 3.3 An Exact Spectral Decomposition

The grouping property of Shannon entropy separates channels exactly. Let $m = p_1$ be the normalized mass of the leading eigenvalue, $q_i = p_i/(1-m)$ for $i \geq 2$ renormalize the rest, and $d_{\perp} = \exp\left(-\sum_{i \geq 2} q_i \log q_i\right)$ be the effective dimensionality of the residual spectrum. With $h(m) = -m\log m - (1-m)\log(1-m)$:

$$
\log d_{\text{eff}} = h(m) + (1-m)\log d_{\perp}\tag{3}
$$

holds to machine precision. Writing $r = \log d_{\perp}$, any trajectory segment is attributed exactly to its three channels by the midpoint split:

$$\Delta \log d_{\text{eff}} = \Delta h(m) - \bar{r}\Delta m + (1-\bar{m})\Delta r$$

The quantity $m$ is called the **common-mode mass** (the leading eigenvector aligns almost perfectly with the cohort-wide agreement direction, squared alignment 0.98), and $r = \log d_{\perp}$ is the **residual dimensionality**.

### 3.4 Statistical Protocol

- The unit of analysis is one (model, question) cohort; budget steps within a cohort are never treated as independent samples
- Headline intervals use **two-way clustered bootstraps** (question × configuration, B=2000)
- Predictive comparisons use **5-fold cross-validation grouped by question**
- RQ3's confirmatory endpoints, decision rule, and verdict sentence were **pre-registered** and Bonferroni-corrected ($\alpha = 0.05/3$)

## Empirical Validation / Results

### RQ1: Temporal Evolution of Routing Effective Rank

**Core trajectory**: Pooled over five benchmarks, $d_{\text{eff}}$ rises from 5.21 at the earliest budget to 18.68 at $b=512$ and returns to 5.84 at the largest budget (5.76 at commit). A prominent interior maximum occurs in **98.5%** of the 3,105 model–question cohorts (95% CI [96.7, 99.8]), with median relative prominence 0.68 of peak height. The rate is 97.9–99.3% on every benchmark and ≥92.9% on every configuration.

**Peak timing** varies systematically:
- Median peak budgets: **768 tokens** (OSS), **1024** (Qwen-Instruct), **4096** (Qwen-Thinking)
- Across datasets, ordering is nearly fixed (Kendall W = 0.946), with GPQA peaking 1.5–2 octaves before math benchmarks

**Controls establish robustness**:
- Permuting expert identities reduces prominent-peak rate from 98.5% to 0.0% and cuts median prominence by factor 178—the arc reflects cross-rollout routing organization, not marginal expert usage
- Restricting to still-generating rollouts preserves interior maximum in 99.8% of cells (ρ = 0.63 with full graph)
- Branch symmetry: same $d_{\text{eff}}$ value can occur on rising and falling branches, making budget context essential

### RQ2: Spectral Anatomy of the Trajectory

**Common-mode alignment**: The leading eigenvector aligns almost perfectly with the graph-wide agreement direction: median $|\langle v_1, u \rangle|^2 = 0.9838$ [0.9800, 0.9872], above 0.8 in 98.3% of frames. Alignment is lowest at the effective-rank maximum (0.970 vs. 0.991 at ends).

**Channel attribution** (over the 16-budget grid):

| Channel | Share of absolute motion | 95% CI |
|---------|--------------------------|--------|
| Common-mode term | 0.666 | [0.645, 0.684] |
| Residual dimensionality | 0.243 | [0.230, 0.260] |
| Mixing | 0.092 | — |

The common-mode term takes 0.63 of the early half and 0.82 of the late half of the budget grid, exceeding the residual term in both.

**Residual dimensionality as complementary axis**: A grouped cross-validated fit of $\log d_{\perp}$ on $m$ explains only **37%** of out-of-sample variance, leaving 63% as a distinct spectral axis.

### RQ3: Behavioral Localization

**Common-mode mass tracks same-answer recoverability** (among non-unanimous cohorts, across 28,622 adjacent-budget steps):
- Dynamic model: $\beta_{\Delta m} = +0.522$ [+0.430, +0.609] (increases in common-mode mass strongly predict recoverability gains)
- Residual coefficient: $\beta_{\Delta r} = -0.047$ [−0.092, −0.016] (opposite-signed, ~quarter the size)
- Within-question time shuffle collapses both coefficients toward zero

**Reasoning effort expands the high-rank regime** (pairing effort tiers within architecture):

| Metric | Effect (higher effort) | 95% CI |
|--------|------------------------|--------|
| Peak delay | +2.59 octaves | [+1.72, +3.83] |
| Peak-normalized occupancy | +0.102 | [+0.045, +0.168] |
| Width of $d_{\text{eff}} \geq 0.8d_{\max}$ | +1.29 octaves | [+0.55, +1.99] |
| Integrated common-mode mass | −0.087 | [−0.154, −0.036] |
| Peak height | +0.104 | [−0.079, +0.232] (spans zero) |

All timing/duration readouts agree directionally in all four architectures, while peak-height contrast varies—localizing the effort effect to **temporal extent rather than peak amplitude**.

## Theoretical and Practical Implications

**Theoretical implications**:
- Establishes that MoE reasoning cohorts follow a **differentiation–reconcentration trajectory**: routing similarity is first concentrated, then spreads across more spectral directions, then reconcentrates—a geometric readout of cohort organization
- Provides an **exact, decomposable measurement framework** that separates common-mode mass from residual dimensionality, turning a visually suggestive curve into interpretable, testable components
- Reveals that the leading eigenmode is a **cohort-wide common mode** (routing agreement in spectral form), giving the familiar graph-density signal a precise spectral interpretation
- The decomposition localizes distinct empirical roles: common-mode governs same-answer recoverability; reasoning effort controls timing/duration of high-rank regime; residual dimensionality retains complementary structural variation

**Practical implications**:
- The diagnostic is **label-free**: computable from router activations alone, before any answer is graded
- Survives frozen transfer to held-out code generation (97.7% prominent-peak rate on 1,582 LiveCodeBench-v5 problems)
- Can be computed from **sparse expert-activation indicators alone** (binary expert-set Jaccard graph) without router gate magnitudes
- Readable online at any token prefix, enabling **serving-time monitoring** of cohort organization
- Provides a principled tool for understanding when reasoning cohorts differentiate, sustain distributed routing structure, and reconcentrate—potentially informing compute allocation and adaptive stopping strategies

## Conclusion

The paper establishes routing effective rank as an **architecture-aware, label-free diagnostic** of cohort routing geometry and temporal organization. The key findings:

1. A **reproducible low–high–low trajectory** across ten MoE configurations and five benchmarks, with peak timing varying systematically with architecture and reasoning effort
2. An **exact decomposition** showing common-mode mass allocation drives most of the motion (two-thirds), while residual dimensionality retains substantial independent structure (one-quarter, with 63% out-of-sample variation beyond common-mode)
3. **Behavioral localization**: common-mode concentration predicts same-answer recoverability; higher reasoning effort delays and broadens the high-rank regime across all four architectures

**Future directions** identified by the authors include:
- Characterizing sensitivity to sampling temperature
- Addressing potential partial entanglement of lexical/formatting cues with same-answer recoverability
- Extending the framework to broader model families and benchmarks
- Exploring the diagnostic's utility in serving-time applications (e.g., compute allocation, adaptive stopping)

The work transforms routing effective rank from a visually suggestive curve into a **decomposable measurement framework** for tracking when MoE reasoning cohorts differentiate, sustain distributed routing structure, and reconcentrate as inference compute unfolds.

---

_Markdown view of https://picx.dev/p/7fVNsz, served by PicX — AI-generated visual whiteboard summaries of research papers._