Back to News

vLLM Blog: FlyDSL MonoKernels Boost Agent Decode on AMD Instinct GPUs

#vllm#amd#decode#agents

The vLLM blog describes how FlyDSL MonoKernels enable high-interactivity decode for agents on AMD Instinct GPUs. The approach uses model-specific fusion, data-ready handoffs, and specialized-decode integration to keep agents responsive.

Coverage timeline

  1. vLLM BlogFelix Li, Lingpeng Jin, Hattie Wu, Carlus Huang, Fangzhou Ai, Zaidy Ali, Amanzhol Salykov, Hyukjoon Lee, Ziming Huang, Jun Chen, Yifeng Huang, Bin Ding, Xiaobing Zhang, Yi Gan, Vinayak Gokhale, Hongxia Yang, Doug Lehr, Richard Li, Andy Luo, Peng Sun, and Emad Barsoum

    How FlyDSL MonoKernels keep agents responsive with high-interactivity decode in vLLM on AMD Instinct GPUs, using model-specific fusion, data-ready handoffs, and specialized-decode integration.

Related stories

  • vLLM Adds Day-0 Support for Flint Sparse Model Checkpoints

    vLLM announced day-0 support for Flint checkpoints, a sparse model with 40 billion total parameters and 2.5 billion active parameters. On a single GPU, vLLM decodes a single request at approximately 500 tokens per second using these checkpoints.

    1 reports · 1 sources
  • GLM-5.3 served at 607.8 tok/s across two Nvidia DGX stations with vLLM

    A hands-on engineering write-up reports serving GLM-5.3 at 607.8 tokens per second across two Nvidia DGX stations using the vLLM inference engine. The post details the setup and performance findings, offering practical insights for high-throughput LLM deployment.

    1 reports · 1 sources149 points · 71 comments
  • FlashStates Hybrid Cache Management Boosts Prefill Throughput Up to 3.14x vs vLLM

    Researchers from Together AI, UIUC, and UT Austin propose FlashStates, a hybrid state cache management system for mixed-architecture LLMs that combine full attention with linear layers. On NVIDIA B200 GPUs, replaying real Claude Code traces, FlashStates achieves up to 3.14x prefill throughput over vLLM and reduces p90 time-to-first-token (TTFT) by up to 88.4%, with benefits persisting when integrated with a 512GiB Mooncake CPU cache.

    1 reports · 1 sources
  • vLLM Serves DeepSeek-V4.1-Flash at 954K Tokens/s per GPU on GB300 NVL72

    vLLM's blog reports that using P/D disaggregation and Wide-EP on a GB300 NVL72 rack, it serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU, matching DeepInfer's performance. This result is 7.9x the aggregated single-node result, demonstrating significant scaling.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe