vLLM and Novita AI Announce Chord, an INT4 MoE Backend for Kimi K2.x
vLLM and Novita AI announced Chord, a faster INT4 MoE backend for Kimi K2.x models, claiming up to 1.3x speedup on H200 and 2.15x on untuned B300. The backend supports uint4, group-32, BF16 scales and the compressed-tensors pack-quantized INT4 group-32 checkpoint format; unsupported quantization schemes fail at load. The distribution provides both chord and humming module roots, with grouped integration with vLLM's Humming backend still in progress.
Coverage timeline
vLLM BlogNovita AI and the vLLM Team
# Do not co-install inclusionAI/humming: Chord intentionally owns that import name (for indexed path, grouped integration is WIP). vllm serve <kimi-k2.x-int4-model> --quantization humming # or select moe_backend="humming" in the vLLM configuration ``` The distribution provides both `chord` and `humming` module roots. vLLM's lazy facade resolves `humming.{dtypes,config,layer,schema,utils.weight}`; the default indexed path can use this existing integration without a Chord-specific framework patch on branches with the WNA16 group-scale support noted below. The shipped schema supports uint4, group-32, BF16 scales and the compressed-tensors pack-quantized INT4 group-32 checkpoint format used by Kimi K2.x; unsupported quantization schemes fail at load instead of silently selecting a wrong kernel. **Grouped integration with vLLM's Humming backend is WIP.** The standalone grouped operator API is shown below. TP8 remains the indexed `h200_tp8` profile because one TP8 weight must serve both phas