Back to News

vLLM: DeepSeek-V4.1-Flash on Vera Rubin NVL72 Hits 58x Throughput vs GB300

#vllm#deepseek#nvidia#throughput

vLLM reported serving DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack, achieving 1.7M tokens/s per GPU and a P90 of 387 tokens/s per user on SemiAnalysis AgentX. This represents 58x the throughput of a GB300 NVL72 at the same interactivity level.

Coverage timeline

  1. vLLM BlogInferact and the vLLM Team
    DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s

    On one NVIDIA Vera Rubin NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 1.7M tokens/s per GPU at a P90 of 387 tokens/s per user, 58x GB300 NVL72 at the same interactivity.

Related stories

  • GLM-5.3 served at 607.8 tok/s across two Nvidia DGX stations with vLLM

    A hands-on engineering write-up reports serving GLM-5.3 at 607.8 tokens per second across two Nvidia DGX stations using the vLLM inference engine. The post details the setup and performance findings, offering practical insights for high-throughput LLM deployment.

    1 reports · 1 sources133 points · 65 comments

  • vLLM Serves DeepSeek-V4.1-Flash at 954K Tokens/s per GPU on GB300 NVL72

    vLLM's blog reports that using P/D disaggregation and Wide-EP on a GB300 NVL72 rack, it serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU, matching DeepInfer's performance. This result is 7.9x the aggregated single-node result, demonstrating significant scaling.

    1 reports · 1 sources

  • DeepSeek releases V4.1-Flash-1017 open-weight model with faster, cheaper agent tasks

    DeepSeek released V4.1-Flash-1017, an open-weight model checkpoint with 763B parameters, now live on its website, app, and API. The update claims stronger benchmark performance, faster task completion, and shorter outputs than its predecessor, achieved via reinforcement learning with self-feedback for end-to-end project development. DeepSeek also opened its Search & Fetch APIs to everyone, priced at $0.75 per 1000 successful requests, and requires DeepSeek Harness v0.2.1-rc.1 or later for optimal use.

    9 reports · 6 sources955 points · 717 comments

  • Llama-modchip runs 284B DeepSeek model on 64GB RAM without GPU

    A project called Llama-modchip demonstrates running a 284B-parameter DeepSeek model on a system with 64GB RAM and no GPU, using a modchip-like approach. The technique was discussed on Hacker News, highlighting a potential path for large-model inference on memory-constrained hardware.

    1 reports · 1 sources483 points · 207 comments

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe