Routing Drift Handling: From Replay to Filtering

Highlights of This Issue

The main theme of this issue is that "the handling of routing consistency is shifting from replay to filtering and control," while server-side reconfiguration begins to exploit the temporal structure exposed by the Attention-FFN Separation (AFD) architecture.

On the training-inference routing consistency side, Router Drift Control for the first time uses response-level routing drift scores as off-policy staleness signals, selectively filtering high-drift responses instead of replaying routes. Unlike R3 (recording and replaying actual routes) in Issue 8's Terminal Agent RL and ESRL (perturbation-replay combination) in Issue 6, this paper does not modify the training routes but uses the inconsistency between rollout and training routes for response-level filtering, and provides a theoretical analysis of gradient bias introduced by route replay—a direct challenge to the burden of proof for route replay methods. It reports complete routing statistics and dense baselines under synchronous/asynchronous RL on Qwen3-30B-A3B.

On the server side, Afore for the first time leverages micro-batch-level demand prefetching and pipeline overlap windows under the AFD architecture, formalizing expert reconfiguration as a micro-batch-aware online scheduling problem, achieving zero exposed migration cost. Compared to Zepp (optimizing communication under relaxed balancing constraints) in Issue 9 and Director (online lookahead placement) in Issue 4, the increment of this paper is that the micro-batch-level real demand exposed by AFD (rather than historical statistics or predictions) directly drives placement decisions. FastEEP advances the granularity of elastic scaling from instance-level to intra-instance dynamic adjustment of EP degree, achieving 4.5–11.7× scaling acceleration through partial state reuse.

On the routing algorithm side, MirrorMoE for the first time systematically reveals the "exploration dilemma" between shared experts and fine-grained expert splitting: shared experts dominate representation learning and reduce routing-induced representation variance, thereby weakening routing exploration, making the combinatorial routing space benefits of fine-grained splitting unrealizable. Smoothing the Top-k Exposure Boundary formalizes the hard selection boundary of static top-k training as a step function of rank exposure, proposing to randomly sample the activated expert budget from a local discrete distribution centered at k during training, softening the boundary without changing the inference mechanism.

Community and Dynamics

Rotate the Router provides a powerful causal control for expert specialization claims: by applying a Haar random orthogonal rotation to the model's own gating (preserving singular values and row norms, only changing gating orientation), it obtains a linear router that "learned nothing" as a reference distribution. On five models, the rotated gating's separation of topics is 17.4–26.0 times higher than its own null hypothesis, indicating that most signals typically reported as specialization are available for any projection; more critically, routing weights are not causally important—the BAR combined router places 62.8% of weights on frozen anchors, with math and code experts totaling only 0.14%, but disabling the math expert loses 30 points on GSM8K while disabling anchors only loses 11.7. This sets a new bar for causal evidence of expert specialization.

Kolibri's Two-Scale Load Balancing advances auxiliary-loss-free bias-based balancing to a two-scale combination: EQB uses precise global quantiles to set expert biases (global balancing), and LEI directly injects the load error of each micro-batch into the router gradient (local balancing), reporting global/local MaxVio and dense baselines at pretraining scale. How a Row Meets Its Experts reports a server-side null result: EPLB's pick-level load balancing does not translate into decoding runtime gains, because decoding time is determined by the number of expert weight bytes read rather than the number of picks—providing a counterexample to the assumption that "balanced routing necessarily brings service benefits."

Fireworks' Numerical Alignment Practice further sinks training-inference mismatch to the numerical layer: on Qwen3.5-MoE, merely matching routing choices is insufficient; mismatches in precision and execution order of expert output aggregation cause approximately 300× k3 drift, and near-tie routing scores can flip expert selection due to rounding—complementing the response-level filtering of Router Drift Control in this issue, suggesting that route replay methods also need alignment at the numerical layer.

Open Questions

1. Router Drift Control filters high-drift responses, while Issue 8's Terminal Agent RL uses R3 to replay actual routes—where is the boundary between filtering and replay? Do discarded high-drift responses contain signals worth learning, or are they purely off-policy noise?

2. Afore relies on micro-batch-level real demand exposed by AFD, while Issue 4's Director uses predictive placement—when AFD is unavailable (e.g., co-located deployments), how can micro-batch-level demand visibility be approximated? How do prediction errors versus real demand gaps affect reconfiguration decisions?

3. MirrorMoE shrinks the exploration space, and Smoothing the Top-k Exposure Boundary softens the top-k boundary—both touch the boundary of routing exploration and exploitation; can they be combined? Does the reduced exploration space weaken the benefits of boundary smoothing?

4. What does the rotated gating control of Rotate the Router mean for the SAE micro-domain proxies of Issue 9's RouterInterp? When specialization signals are available for any projection, does the discriminative power of SAE features as micro-domain proxies need recalibration?