The Value of Dynamic Expert Pruning Is Budget-Dependent

Highlights of This Issue

This issue's main thread continues the discussion on 'controlled baselines' from the previous issue and advances the conclusion one step further: the value of dynamic expert pruning is not binary (present or absent) but budget-dependent. Issue 11's 'Don't Skip the Baseline' established a static baseline that matches the measured compute. This issue's You Only Need 2/3 of the Chosen Experts extends this burden of proof to fine-grained MoE across nine architectures and twelve checkpoints: uniform truncation retaining ⌈2K/3⌉ experts on average retains 98.8% performance, dynamic rules under conservative budgets differ from uniform truncation by only +0.06 points on average, and advantages appear only under aggressive pruning (≤1/2 of native K), with gains up to +3.0 points, while the winner rotates over time. Compared to Issue 11, this paper adds task family (generation vs. likelihood) and model attribute (thinking/scale/multimodal) decompositions of pruning sensitivity, advancing the conclusion from a binary outcome to a budget-bound decision; its candid limitation of DiEP to implementing only the skipping criterion rather than a differentiable pruning objective, and its conservative protocol of taking the low value for the Fixed-K reference line, also provide a rigorous template reusable for evaluation in this direction.

The second line is about 'bias-based balancing mechanisms without auxiliary loss being dissected and reused.' Three works independently approach this from three angles. Muon Makes the First Mixture-of-Experts Layer Harder to Balance is the first to measure the impact of the optimizer (Muon vs. AdamW) on the workload of bias balancers, finding that Muon doubles the bias span in the first MoE layer, attributing it to shared directions induced by the backbone rather than the router itself. On the community side, Dual-Bias Load Balancing addresses shallow-layer bias lag by introducing intra-step micro-bias residuals. Overcoming Data Imbalances with Expert Selection Biases reuses bias from load signal to class-conditional capacity allocation, proposing preferred expert share to expose the size of the expert pool per class independently of data share as an explicit knob, and provides qualified causal evidence for expert specialization via functional modular intervention (disabling Swahili-specified experts causes +107% loss vs. +9.3% baseline).

On the serving and scaling side, Rimuru unifies expert caches with KV/SSM states into a shared page pool, enabling cross-type elastic capacity reuse through multi-view memory layouts and page-table-aware fused kernels, offering a new system-level path for elastic expert caching in serving. Multi-View MoE extends the differentiated preferences of multi-routing heads from pure selection signals to expert input construction, yielding gains without increasing the number of active experts, expert FLOPs, or all-to-all communication. Meanwhile, two opposing voices emerge in the community: EMNLP's unified scaling law and granularity expressiveness theory provide positive arguments for fine-grained experts, while 'The MoE Mirage' presents counterexamples to the free-lunch narrative of sparse activation on consumer-grade hardware.

Community and Dynamics

Towards a Comprehensive Scaling Law of Mixture-of-Experts (EMNLP 2026) provides a unified scaling law jointly characterizing granularity G, sparsity Na/N, and shared expert ratio S, yielding a scaling-invariant near-optimal G≈6.8 and noting that optimal activation ratio becomes sparser as model size grows, offering a unified framework for sparsity-granularity scaling that dialogues with Issue 3's 'Slicing and Dicing.' Granularity boosts expressivity in mixture of experts provides a theoretical anchor: proving that the number of configurations C(m,k), determined by the number of active experts, yields exponential expressivity separation, offering a formal argument for fine-grained experts independent of sampling, and pointing out that communication and routing costs from high granularity remain an open problem.

MoE Load Balancing (EPLB) (xLLM engineering docs) demonstrates a productionized form of online redundant expert rebalancing: layer-wise weight migration and asynchronous prepare protocol. As a system description rather than a research contribution, it provides an industry reference form for 'online rebalancing of redundant experts,' but lacks experimental ablations and routing statistics. The MoE Mirage is an opinionated counterexample on the cost-effectiveness of sparse activation: on consumer/edge hardware, cost grows with total parameters rather than active parameters, directly challenging the sparse-acceleration assumption outside data centers and worth validating as a boundary condition for inference efficiency narratives.

Open Questions

1. Issue 11's 'Don't Skip the Baseline' shows that K_eff calibration deviation can be up to 6-fold, while this issue's 'You Only Need 2/3' demonstrates that the value of dynamic rules exactly appears in the aggressive budget region, which is where calibration is most fragile. After calibration errors are matched, does dynamic allocation retain remaining value? Can budget-dependent conclusions be reduced to a more single mechanism (e.g., generation tasks being more sensitive to token-wise allocation)?

2. Muon doubles the bias span in the first MoE layer, while Dual-Bias exactly targets shallow-layer bias lag — is the bias-based balancing mechanism itself a weak link under training dynamics? Are optimizer-induced bias growth and shallow-layer stagnation homologous? Can the three facets of bias mechanism (load, stability, capacity allocation) be unified into one control variable?

3. Overcoming Data Imbalances with Expert Selection Biases decouples the class expert pool from data share (r_c up to 4), but what is the upper limit of r_c? Will it hit the hard constraint of per-token active number K, or cause negative transfer across classes? After removing inference-time training bias, does it still hold under RL and post-training stability?

4. After Rimuru unifies expert weights and KV states into a shared page pool, will elastic expert offloading interact with routing stability and training-inference consistency — when active experts are swapped out of memory, does it introduce a new source of train-rollout mismatch?