Back to News

vLLM Adds Day-0 Support for Kimi K3.1 Pro with Sparse Attention and Packed KV Cache

#vllm#kimi#sparse attention#kv cache

vLLM announced day-0 support for Kimi K3.1 Pro, enabling sparse attention on a KDA hybrid, a packed KV cache with FP8 or NVFP4 latents and MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode. This integration allows users to run the model immediately upon release with optimized performance.

Coverage timeline

  1. vLLM BlogInferact and the vLLM Team
    vLLM Delivers Day-0 Support for Kimi K3.1 Pro

    vLLM supports Kimi K3.1 Pro on day 0: sparse attention on a KDA hybrid, a packed KV cache with FP8 or NVFP4 latents and MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode.

Related stories

  • Moonshot unveils Kimi K3.1 Pro, 3T open-weight model with 1M context, weights due Nov 4

    Moonshot AI announced Kimi K3.1 Pro, a 3-trillion-parameter open-weight reasoning model with a 1M-token context window, alongside a faster Highspeed variant. Full weights are promised by November 4, 2026. The model scores 55 on the Artificial Analysis Intelligence Index, ranking 4th overall and first among open-weight models, with API pricing at $2/M input and $10/M output tokens.

    6 reports · 4 sources

  • vLLM Adds Day-Zero Support for Mistral Large 4 on ROCm with TP8/TP4

    vLLM announced day-zero support for serving Mistral Large 4 (mistralai/Mistral-Large-4-1T-A52B) on AMD ROCm, with configurations for tensor-parallel size 8 (one replica per node) and tensor-parallel size 4 (two replicas per node). The setup uses AITER optimizations, FP8 MoE requantization (mxfp4), speculative decoding with dspark, and tool-call/reasoning parsers.

    1 reports · 1 sources

  • vLLM Adds Day-0 Support for NVIDIA Vera Rubin NVL72, Claims 7.8x Throughput over GB200

    vLLM announced support for NVIDIA Vera Rubin NVL72, claiming 7.8x throughput over GB200 NVL72. The platform offers daily container builds and day-0 model support for DeepSeek, Moonshot AI, Z.ai, and MiniMax, leveraging Blackwell kernel compatibility and Rubin-tuned kernels via FlashInfer 0.7.0.

    1 reports · 1 sources

  • vLLM Adds Day-0 Support for Flint Sparse Model Checkpoints

    vLLM announced day-0 support for Flint checkpoints, a sparse model with 40 billion total parameters and 2.5 billion active parameters. On a single GPU, vLLM decodes a single request at approximately 500 tokens per second using these checkpoints.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe