Summary (Overview)

• The paper introduces routing effective rank ($d_{\text{eff}}$), a novel label-free diagnostic that measures the entropy-effective dimensionality of a cross-rollout graph built from Mixture-of-Experts (MoE) expert-routing similarity, providing a spectral lens on how reasoning cohorts reorganize during test-time inference.
• Across ten MoE configurations, five math/science benchmarks, and 3,105 model–question cohorts, $d_{\text{eff}}$ exhibits a reproducible low–high–low trajectory with a prominent interior maximum in 98.5% of cohorts: routing similarity is concentrated early, maximally differentiated at intermediate budgets, and reconcentrated later.
• An exact spectral decomposition ($\log d_{\text{eff}} = h(m) + (1-m)\log d_{\perp}$) separates cohort-wide common-mode mass from residual spectral dimensionality, revealing that common-mode reallocation accounts for ~two-thirds of the trajectory while residual spectrum contributes ~one-quarter with substantial independent variation (grouped-CV $R^2 = 0.37$).
• Behavioral localization shows: common-mode concentration strongly predicts same-answer recoverability (β = +0.522), and higher reasoning effort delays the peak by +2.59 octaves (doublings of token budget) and widens the high-rank regime across all four tested architectures—localizing the effort effect in timing and duration rather than peak amplitude.
• The diagnostic is practical: it survives frozen transfer to held-out code generation (97.7% peak rate on LiveCodeBench-v5) and can be computed from sparse expert-activation indicators alone, making it applicable during serving before any answer is graded.

Introduction and Theoretical Foundation

Test-time scaling transforms inference into a population process: instead of one answer, a model produces a cohort of candidate reasoning trajectories combined via majority vote, verifiers, or learned rewards. While most existing analyses characterize cohorts through final answers (votes, margins, verifier scores), the paper argues that before answers are compared, the cohort already has internal organization—some trajectories compute similarly, others route through different expert patterns.

Sparse Mixture-of-Experts (MoE) models expose this organization through routing traces, which are by-products of decoding (arriving with every token) rather than added probes. Comparing traces across same-question rollouts yields an activation-only similarity graph whose spectrum compactly describes cohort geometry.

The central quantity is routing effective rank $d_{\text{eff}}$, the exponential of spectral entropy applied to the routing-similarity spectrum—the same functional as the Vendi score on similarity matrices. Unlike ordinary rank, it discounts numerically tiny directions; unlike graph density, it summarizes the full eigenvalue distribution.

The paper addresses three research questions:

• RQ1: Temporal regularity and architecture-dependent timing of the trajectory
• RQ2: The spectral channels (common-mode vs. residual) that account for the trajectory
• RQ3: How those channels relate to answer organization and reasoning effort

Key theoretical insight: The interpretation of $d_{\text{eff}}$ depends on how spectral mass is allocated—a rise may reflect a weaker cohort-wide routing mode, a richer residual spectrum, or both. This motivates studying $d_{\text{eff}}$ as a decomposable trajectory rather than an isolated scalar.

Methodology

3.1 Cohorts and the Routing Graph

For each question, a cohort of $N=64$ rollouts is sampled and evaluated along a logarithmic budget axis of token prefixes $b \in \{32, 48, \dots, 32768\}$ (16 points), plus a per-rollout commit window at each boxed final answer. At budget $b$, each rollout $i$ has a histogram $h_i(b)$ accumulating the router's gate weight over each (layer, expert) slot in a short trailing window (16 tokens by default).

The symmetric cohort graph $\mathbf{W} \in \mathbb{R}^{N \times N}$ holds weighted Jaccard similarity:

$$\mathbf{W}_{ij}(b) = \frac{\sum_{e} \min\bigl(h_i^{(e)}(b), h_j^{(e)}(b)\bigr)}{\sum_{e} \max\bigl(h_i^{(e)}(b), h_j^{(e)}(b)\bigr)},\tag{1}$$

indexed over expert slots $e$. Edge weights are computed from router activations only; answer annotations never enter the similarity.

3.2 Routing Effective Rank

With $p_i = \lambda_i / \sum_j \lambda_j$ over the eigenvalues of the symmetrized $\mathbf{W}$, the effective rank is:

$$d_{\text{eff}}(\mathbf{W}) = \exp\Bigl(-\sum_{i} p_i \log p_i\Bigr),\tag{2}$$

the entropy-effective dimensionality of the routing-similarity spectrum. Under approximately block-structured geometry, it approximates an effective number of balanced routing groups; more generally, it quantifies spectral dimensionality rather than a literal cluster count.

3.3 An Exact Spectral Decomposition

The grouping property of Shannon entropy separates channels exactly. Let $m = p_1$ be the normalized mass of the leading eigenvalue, $q_i = p_i/(1-m)$ for $i \geq 2$ renormalize the rest, and $d_{\perp} = \exp\left(-\sum_{i \geq 2} q_i \log q_i\right)$ be the effective dimensionality of the residual spectrum. With $h(m) = -m\log m - (1-m)\log(1-m)$:

$$\log d_{\text{eff}} = h(m) + (1-m)\log d_{\perp}\tag{3}$$

holds to machine precision. Writing $r = \log d_{\perp}$, any trajectory segment is attributed exactly to its three channels by the midpoint split:

$$\Delta \log d_{\text{eff}} = \Delta h(m) - \bar{r}\Delta m + (1-\bar{m})\Delta r$$

The quantity $m$ is called the common-mode mass (the leading eigenvector aligns almost perfectly with the cohort-wide agreement direction, squared alignment 0.98), and $r = \log d_{\perp}$ is the residual dimensionality.

3.4 Statistical Protocol

• The unit of analysis is one (model, question) cohort; budget steps within a cohort are never treated as independent samples
• Headline intervals use two-way clustered bootstraps (question × configuration, B=2000)
• Predictive comparisons use 5-fold cross-validation grouped by question
• RQ3's confirmatory endpoints, decision rule, and verdict sentence were pre-registered and Bonferroni-corrected ($\alpha = 0.05/3$)

Empirical Validation / Results

RQ1: Temporal Evolution of Routing Effective Rank

Core trajectory: Pooled over five benchmarks, $d_{\text{eff}}$ rises from 5.21 at the earliest budget to 18.68 at $b=512$ and returns to 5.84 at the largest budget (5.76 at commit). A prominent interior maximum occurs in 98.5% of the 3,105 model–question cohorts (95% CI [96.7, 99.8]), with median relative prominence 0.68 of peak height. The rate is 97.9–99.3% on every benchmark and ≥92.9% on every configuration.

Peak timing varies systematically:

• Median peak budgets: 768 tokens (OSS), 1024 (Qwen-Instruct), 4096 (Qwen-Thinking)
• Across datasets, ordering is nearly fixed (Kendall W = 0.946), with GPQA peaking 1.5–2 octaves before math benchmarks

Controls establish robustness:

• Permuting expert identities reduces prominent-peak rate from 98.5% to 0.0% and cuts median prominence by factor 178—the arc reflects cross-rollout routing organization, not marginal expert usage
• Restricting to still-generating rollouts preserves interior maximum in 99.8% of cells (ρ = 0.63 with full graph)
• Branch symmetry: same $d_{\text{eff}}$ value can occur on rising and falling branches, making budget context essential

RQ2: Spectral Anatomy of the Trajectory

Common-mode alignment: The leading eigenvector aligns almost perfectly with the graph-wide agreement direction: median $|\langle v_1, u \rangle|^2 = 0.9838$ [0.9800, 0.9872], above 0.8 in 98.3% of frames. Alignment is lowest at the effective-rank maximum (0.970 vs. 0.991 at ends).

Channel attribution (over the 16-budget grid):

The common-mode term takes 0.63 of the early half and 0.82 of the late half of the budget grid, exceeding the residual term in both.

Residual dimensionality as complementary axis: A grouped cross-validated fit of $\log d_{\perp}$ on $m$ explains only 37% of out-of-sample variance, leaving 63% as a distinct spectral axis.

RQ3: Behavioral Localization

Common-mode mass tracks same-answer recoverability (among non-unanimous cohorts, across 28,622 adjacent-budget steps):

• Dynamic model: $\beta_{\Delta m} = +0.522$ [+0.430, +0.609] (increases in common-mode mass strongly predict recoverability gains)
• Residual coefficient: $\beta_{\Delta r} = -0.047$ [−0.092, −0.016] (opposite-signed, ~quarter the size)
• Within-question time shuffle collapses both coefficients toward zero

Reasoning effort expands the high-rank regime (pairing effort tiers within architecture):

All timing/duration readouts agree directionally in all four architectures, while peak-height contrast varies—localizing the effort effect to temporal extent rather than peak amplitude.

Theoretical and Practical Implications

Theoretical implications:

• Establishes that MoE reasoning cohorts follow a differentiation–reconcentration trajectory: routing similarity is first concentrated, then spreads across more spectral directions, then reconcentrates—a geometric readout of cohort organization
• Provides an exact, decomposable measurement framework that separates common-mode mass from residual dimensionality, turning a visually suggestive curve into interpretable, testable components
• Reveals that the leading eigenmode is a cohort-wide common mode (routing agreement in spectral form), giving the familiar graph-density signal a precise spectral interpretation
• The decomposition localizes distinct empirical roles: common-mode governs same-answer recoverability; reasoning effort controls timing/duration of high-rank regime; residual dimensionality retains complementary structural variation

Practical implications:

• The diagnostic is label-free: computable from router activations alone, before any answer is graded
• Survives frozen transfer to held-out code generation (97.7% prominent-peak rate on 1,582 LiveCodeBench-v5 problems)
• Can be computed from sparse expert-activation indicators alone (binary expert-set Jaccard graph) without router gate magnitudes
• Readable online at any token prefix, enabling serving-time monitoring of cohort organization
• Provides a principled tool for understanding when reasoning cohorts differentiate, sustain distributed routing structure, and reconcentrate—potentially informing compute allocation and adaptive stopping strategies

Conclusion

The paper establishes routing effective rank as an architecture-aware, label-free diagnostic of cohort routing geometry and temporal organization. The key findings:

1. A reproducible low–high–low trajectory across ten MoE configurations and five benchmarks, with peak timing varying systematically with architecture and reasoning effort
2. An exact decomposition showing common-mode mass allocation drives most of the motion (two-thirds), while residual dimensionality retains substantial independent structure (one-quarter, with 63% out-of-sample variation beyond common-mode)
3. Behavioral localization: common-mode concentration predicts same-answer recoverability; higher reasoning effort delays and broadens the high-rank regime across all four architectures

Future directions identified by the authors include:

• Characterizing sensitivity to sampling temperature
• Addressing potential partial entanglement of lexical/formatting cues with same-answer recoverability
• Extending the framework to broader model families and benchmarks
• Exploring the diagnostic's utility in serving-time applications (e.g., compute allocation, adaptive stopping)

The work transforms routing effective rank from a visually suggestive curve into a decomposable measurement framework for tracking when MoE reasoning cohorts differentiate, sustain distributed routing structure, and reconcentrate as inference compute unfolds.