# SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving

> Expert reduction in MoE serving weakens model refusal by up to 18.4 percentage points, but running all experts for the prompt and first 8 tokens restores safety while keeping most speedup.

- **Source:** [arXiv](https://arxiv.org/abs/2611.06147)
- **Published:** 2026-11-07
- **Permalink:** https://picx.dev/p/7LdKuq
- **Whiteboard:** https://picx.dev/p/7LdKuq/image

## Summary

## Summary (Overview)

- **Core problem identified**: Expert reduction methods in Mixture-of-Experts (MoE) language model serving—which run fewer experts per token for speed—can significantly weaken a model's refusal of harmful requests, even while accuracy benchmarks remain flat.
- **Key finding on where safety breaks**: The refusal degradation arises primarily during prompt processing (prefill) and the first few generated tokens, not uniformly across generation.
- **SafeStart solution**: A vLLM engine patch that runs all experts for the prompt and first N=8 generated tokens, then applies the reduced expert budget afterward, restoring refusal while keeping most of the speedup.
- **Quantified impact**: Under a refusal-suppression jailbreak, expert truncation raises harmful compliance by up to 18.4 percentage points (pp) on affected models while MMLU accuracy drops by less than 2 points; SafeStart brings the change to between −2.1 and +1.3 pp.
- **Speed preservation**: SafeStart keeps at least 85% of the offline decode speedup at batch sizes 8 and 32, and with half the experts achieves 38–44% faster decoding than the original model at batch size 8.

---

## Introduction and Theoretical Foundation

### Background

Mixture-of-Experts (MoE) language models route each token through a small subset of many expert subnetworks. Serving methods accelerate these models by running even fewer experts per token, validated primarily on accuracy benchmarks. In vLLM and llama.cpp, expert truncation is a one-line configuration change.

### Theoretical Basis

**MoE layer output** (Equation 1):
$$y(x) = g_s(x) f_s(x) + \sum_{i \in \mathcal{T}_k(x)} w_i(x) f_i(x)$$

where:
- $f_s$: shared experts (present in some models)
- $g_s$: sigmoid gate (Qwen1.5-MoE, Qwen2-57B) or constant 1 (DeepSeek-V2-Lite, others)
- $\mathcal{T}_k(x)$: set of $k$ highest-scoring experts selected by the router
- $w_i(x)$: router weight for expert $i$
- $p(x) = \text{softmax}(W_r x)$: router scores (Moonlight uses sigmoid with learned bias)

**Gate normalization matters**: With top-k renormalization, weights $w_i = p_i / \sum_{j \in \mathcal{T}_k} p_j$ sum to one. Without it, $w_i = p_i$ and routed experts carry weight $m_k(x) = \sum_{j \in \mathcal{T}_k} p_j < 1$ (gate mass). Removing experts without renormalization lowers gate mass, reducing routed expert contribution.

### Key Insight

Refusal in aligned models is fragile—it depends on a few experts and routers whose removal leaves accuracy largely intact. Prior work shows alignment changes mainly affect the first tokens of a response. No prior work tested expert reduction at published operating points for refusal, located where refusal changes inside a serving engine, and fixed it there while keeping speedup.

---

## Methodology

### Threat Model

An operator serves an aligned MoE model and enables an expert-reduction method validated on accuracy benchmarks. A user (without access to weights, routing, or configuration) submits harmful requests, optionally wrapped in published jailbreak templates.

### Models and Prompts

- **Models**: Nine MoE models (Qwen1.5-MoE, DeepSeek-V2-Lite, Moonlight-16B, Ling-lite-1.5, OLMoE-1B-7B, Qwen3-30B-A3B, GPT-OSS-20B, plus held-out DeepSeek-MoE-16B and Qwen2-57B-A14B)
- **Prompts**: 159 HarmBench standard behaviors + 313 StrongREJECT prompts = 472 total, presented in multiple framings (plain, refusal suppression, prefix injection, plus robustness checks)
- **Main operating point**: $k' = \lceil 2k/3 \rceil$ (about two-thirds of selected experts)

### Measuring Refusal

- **Primary judge**: Llama Guard 3 8B labels responses as safe/harmful
- **Secondary judge**: HarmBench classifier
- **Human validation**: Two blind annotators on 587 responses (97.3% agreement, κ = 0.93)
- **Paired statistics**: Exact McNemar test on forward/reverse flips; Tango 95% intervals; equivalence margin of ±3 pp
- **Accuracy checks**: MMLU, GSM8K, XSTest (over-refusal)

### Measuring Speed

- Single NVIDIA RTX A6000 GPU
- Offline: decode time for 256 tokens per request, batch sizes 1–256
- Online: median time per output token under Poisson arrivals
- Speedup explained by counting distinct expert weight reads per decode step

### SafeStart Implementation

A ~160-line Python/Triton patch to vLLM 0.11.0:
1. Marks tokens at position $P - 1 + N$ or later in a GPU buffer read by CUDA graphs
2. A fused Triton kernel replaces slots outside the reduced budget with a sentinel expert of weight zero (applies top-k′, top-p, or threshold budgets)
3. vLLM's expert map sends the sentinel to −1, so the expert kernel skips its weights

---

## Empirical Validation / Results

### Speedup Mechanism

Truncation saves time by reducing expert weight reads from GPU memory. At batch size 8, decode speedup is 15–28% on all six timed models; at batch size 128, at most 5% on four models. The speedup peaks at batch size 8 because larger batches select almost every expert anyway.

### Refusal Degradation (Table 2, confirmatory tests)

Under refusal suppression, truncation at $k' = \lceil 2k/3 \rceil$ raises ASR by:

| Model | Original ASR | Δ ASR (pp) | MMLU drop |
|-------|-------------|-------------|-----------|
| Qwen1.5-MoE | 46.6% | **+18.4** | ≤1.8 |
| OLMoE-0125 | 24.2% | **+9.5** | ≤1.8 |
| DeepSeek-V2-Lite | 43.6% | **+6.6** | ≤1.8 |

Human labels confirm: +13.6 pp on Qwen1.5 for clearly harmful responses; +17.0 pp including partial compliance.

### Where the Change Arises

- 92 of 101 responses that truncation turns harmful on Qwen1.5 differ from the original already at the first generated token
- Truncating only the prompt and first token reproduces 91% of the increase on Qwen1.5 and 62% on OLMoE
- SafeStart with N=1 (protecting prompt + first token) leaves at most 0.9 pp on all three models
- A refusal direction carries the change: restoring the original position along this direction removes 97% (Qwen1.5) and 71% (OLMoE) of the first-token increase

### SafeStart Results (Table 3)

With N=8, SafeStart changes ASR under refusal suppression by −0.6, +0.6, and +1.3 pp on Qwen1.5, DeepSeek-V2-Lite, and OLMoE (vs. +18.4, +6.6, +9.5 pp for truncation). Pooled change: +0.4 pp (90% CI [−0.6, 1.6]), within the ±3 pp equivalence margin.

### Held-out Validation

On 889 held-out requests (AdvBench + CatQA), truncation raises ASR by 5.8–21.4 pp; SafeStart leaves a rise of 0.6–2.6 pp, non-inferior to the original model on two of three models.

### Speed Preservation

- With no expert removed, the patch adds at most 2.4% to median time per output token
- SafeStart keeps 85–101% of truncation's decode speedup at batch size 8, 88–104% at batch size 32
- With half the experts: 38–44% faster than original at batch size 8, with GSM8K scores 3.6–3.9 points higher than truncation

### Other Reduction Methods

- **Top-p routing**: raises ASR by up to 30.1 pp; SafeStart brings to −2.1 pp
- **LExI** (layer-wise expert allocation): SafeStart reduces ASR from 51.8% to 36.4% on Qwen1.5 (original: 35.7%)
- **ACE** (per-token skipping): at ~50% skipping, SafeStart reduces ASR change from +21.0 to −0.4 pp on Qwen1.5

---

## Theoretical and Practical Implications

### For Serving Systems

1. **Test refusal, not only accuracy**: A one-line config change at a recommended operating point raises harmful compliance by up to 18.4 pp while MMLU moves ≤1.8 points. Serving systems must test refusal per model and method.
2. **Reduce experts only where it pays**: Truncation helps at small batch sizes and provides almost no speedup at large batches—where it adds refusal cost for nothing.

### For Expert-Reduction Methods

- Methods should report refusal separately for prefill and decoding
- The computation of prompt and first generated tokens decides most of the safety change
- SafeStart's approach (protecting the opening) generalizes across top-k, top-p, threshold skipping, LExI, and ACE

### For Safety Research

- Confirms refusal is mediated by a small set of experts/routers in MoE models
- Demonstrates that serving-level optimizations can silently undermine alignment
- Provides a practical, low-overhead fix inside the serving engine without extra models

---

## Conclusion

### Main Takeaways

1. **Faster MoE serving need not cost safety**: Running all experts at the start of each response preserves most of the speedup from expert reduction while restoring refusal behavior.
2. **The problem is detectable and fixable**: Most of the refusal change arises in prefill and the first few generated tokens, making targeted protection effective.
3. **Safety evaluation must be paired and position-aware**: Accuracy benchmarks are insufficient; refusal must be tested at the actual expert budgets tokens receive.

### Future Directions

- Multi-turn serving (must add expert budget to prefix-cache key)
- Quantized MoE kernels and expert parallelism support
- Adaptive attacks against SafeStart's protected opening
- Larger models (>57B parameters) and additional hardware configurations

### Limitations

- Human labels cover only 587 responses from two models
- Templates that dictate opening words or move harmful tasks later can bypass SafeStart
- The first token does not carry the increase on one held-out model (DeepSeek-MoE-16B)
- The engine patch is single-turn and does not cover all deployment configurations

**Bottom line**: SafeStart demonstrates that a serving system can keep most of the gain from running fewer experts by running all of them at the start of each response, and it should test refusal—not only accuracy—before enabling such methods.

---

_Markdown view of https://picx.dev/p/7LdKuq, served by PicX — AI-generated visual whiteboard summaries of research papers._