Back to News

vLLM Adds Day-0 Support for Flint Sparse Model Checkpoints

#vllm#flint#sparse-model#inference

vLLM announced day-0 support for Flint checkpoints, a sparse model with 40 billion total parameters and 2.5 billion active parameters. On a single GPU, vLLM decodes a single request at approximately 500 tokens per second using these checkpoints.

Coverage timeline

  1. vLLM BlogInferact and Recursive
    Day-0 Support for Flint on vLLM

    vLLM supports both Flint checkpoints (40B total, 2.5B active) from day 0, decoding a single request at about 500 tokens/s on one GPU.

Related stories

  • Reflection AI releases Apache 2.0-licensed Beam, a 501B-parameter open model

    Reflection AI released the weights of Beam, its first open model, under the Apache 2.0 license. Beam has 501B total parameters with 23B active, and was trained with reinforcement learning that penalized unnecessary tokens. vLLM announced day-0 support for serving Beam, with agent-driven optimization cutting decode latency by 2.71x and DFlash2 speculative decoding with adaptive verification.

    2 reports · 2 sources

  • vLLM Adds Day-0 Support for NVIDIA Vera Rubin NVL72, Claims 7.8x Throughput over GB200

    vLLM announced support for NVIDIA Vera Rubin NVL72, claiming 7.8x throughput over GB200 NVL72. The platform offers daily container builds and day-0 model support for DeepSeek, Moonshot AI, Z.ai, and MiniMax, leveraging Blackwell kernel compatibility and Rubin-tuned kernels via FlashInfer 0.7.0.

    1 reports · 1 sources

  • NVIDIA launches Nemotron 3.5 Super VL, 120B open model; vLLM adds Day 0 support

    NVIDIA has launched Nemotron 3.5 Super VL, a 120B-parameter open model for enterprise agents, claiming it outperforms frontier models on specialized benchmarks. vLLM announced Day 0 support for the model, enabling multimodal agent workflows with image, document, and video understanding, OpenAI-compatible serving, and multi-token prediction.

    2 reports · 2 sources

  • vLLM Uses AI Agents to Optimize Kimi K3 Decode on NVIDIA GB300, Hitting 840+ Tokens/s

    The vLLM team reports using agent-assisted development to build a unified CuTe DSL decode path for Kimi K3 on NVIDIA GB300, covering attention, MoE, and communication across 8 or 16 GPUs. The optimizations achieve over 840 tokens/s at batch 1 on 16 GPUs with DSpark (synthetic AL=6), 1.5–2× higher decode throughput in several configurations, and up to 1.8× higher P90 interactivity on the AgentX workload. The team notes that agent-optimized inference code is outpacing their manual review and maintenance workflow.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe