vLLM reports PD serving results for Qwen3.8-2.4T on GB300 NVL72
vLLM's blog reports PD serving performance for the Qwen3.8-2.4T model on a GB300 NVL72 cluster with an 8K/1K workload. In high-throughput scenarios, vLLM achieved 5000 total token throughput per GPU, and 180 generated tokens per user in low-latency scenarios, both on the Pareto frontier. The post provides srt-slurm recipes and details the decision-making process for reproduction and optimization.
Coverage timeline
vLLM BlogvLLM Team
## TL;DR In this blog post we present our latest performance results on PD serving of the Qwen3.8-2.4T model using vLLM on a GB300 NVL72 cluster on 8K/1K workload. In high throughput scenario vLLM achieved **5000** total token throughput per GPU, and **180** generated tokens per user in the low latency scenario — both presented on pareto frontier below. In this blog post we explain in detail how we achieved these results, and give precise srt-slurm recipes to allow anyone to verify and reproduce them using local serving. But what is more important — we describe our decision making process we used to create these recipes. This is even more important than the actual performance results, since it allows you to squeeze performance out of PD serving on vLLM for any model you desire. ## Introduction Recently, we presented our Qwen3.5 PD serving results and showed that vLLM could reach 25K Total TPS/GPU. Straight after that, a new frontier Qwen-family model was released: Qwen3.8-2.4T. In this