GLM-5.3 served at 607.8 tok/s across two Nvidia DGX stations with vLLM
A hands-on engineering write-up reports serving GLM-5.3 at 607.8 tokens per second across two Nvidia DGX stations using the vLLM inference engine. The post details the setup and performance findings, offering practical insights for high-throughput LLM deployment.
Coverage timeline
Hacker Newswondersky
Related stories
vLLM Serves DeepSeek-V4.1-Flash at 954K Tokens/s per GPU on GB300 NVL72
vLLM's blog reports that using P/D disaggregation and Wide-EP on a GB300 NVL72 rack, it serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU, matching DeepInfer's performance. This result is 7.9x the aggregated single-node result, demonstrating significant scaling.
1 reports · 1 sources
vLLM: DeepSeek-V4.1-Flash on Vera Rubin NVL72 Hits 58x Throughput vs GB300
vLLM reported serving DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack, achieving 1.7M tokens/s per GPU and a P90 of 387 tokens/s per user on SemiAnalysis AgentX. This represents 58x the throughput of a GB300 NVL72 at the same interactivity level.
vLLM Optimizes GLM-5.3 Serving on 16 H200s with PCP, DCP, Hybrid HiSparse
vLLM's blog details serving GLM-5.3 on 16 H200 GPUs using PCP and DCP for prefill and Hybrid HiSparse for decode, enabling 1M-token contexts. On the AgentX agentic benchmark, this configuration achieves 1.7–3.2× per-user speedups over data-parallel prefill/decode.
vLLM Adds Day-0 Support for Flint Sparse Model Checkpoints
vLLM announced day-0 support for Flint checkpoints, a sparse model with 40 billion total parameters and 2.5 billion active parameters. On a single GPU, vLLM decodes a single request at approximately 500 tokens per second using these checkpoints.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe