vLLM-Omni and FastVideo Optimize MiniMax H3 Serving, Generate MP4 in 8.7s on Eight B300 GPUs
A technical post from vLLM-Omni and FastVideo describes system-level optimizations for serving MiniMax H3, a multimodal model that generates video with synchronized audio. The optimizations span the full resident pipeline, including attention and communication, fused DiT operators, parallel VAE decoding, compact output transfer, and parallel MP4 construction. FastVideo's FastH3 reduces 49 DiT forward passes to 4, achieving real-time full MP4 generation in 8.678–8.710 seconds on eight B300 GPUs for a 10.125-second clip.
Coverage timeline
vLLM BlogAMD and Embedded LLM Teams
A performance model for LLM serving: inspect local shapes, remove repeated work, verify data movement and dispatch, then follow the queue.
机器之心机器之心
MiniMax H3 的服务是一个系统问题。一个请求要经过一个庞大的 Qwen3-VL 编码器、一个长序列的音视频 DiT、相互独立的视频与音频 VAE、设备与进程的边界，最后还要完成 H.264/AAC 的封装。只优化 DiT，其余环节的时延依然留在那里。 因此 vLLM-Omni 从完整的常驻流水线入手：注意力与通信、融合的 DiT 算子、并行 VAE 解码、紧凑的输出传输，以及并行的 MP4 构建。随后由 FastVideo 的 FastH3 处理剩下的主导项，把 49 次 DiT 前向替换为 4 次。 在实测的八卡 B300 配置上，FastH3 产出一个 10.125 秒的完整 MP4 用时 8.678 至 8.710 秒 。本文中的 实时 指完整响应的就绪时间快于其播放时长，并不指流式交付或首帧时间。 1. 为什么 MiniMax H3 的服务是一个系统级问题 MiniMax H3 以文本、图像、视频和音频作为输入，联合生成视频与同步音频。它的各个组件在算力、显存和放置上的要求各不相同： request -> encoder -> joint audio/video DiT -> video + audio VAEs -> GPU output preparation -> D2H/IPC -> H.264/AAC MP4 图 1：文本走 H3/Qwen3-VL 编码器；视觉与音频条件还会分别经过各自的 VAE。条件潜变量与带噪的目标潜变量组成一条打包序列，进行联合的音视频去噪，之后再分别解码并构建 MP4。 已发布的 checkpoint 覆盖三种服务任务： DiT 主导了基础调度，但它并不是唯一的瓶颈。编码器的常驻会影响容量；去噪被缩短之后，VAE 解码就会显现出来；而原始帧仍然必须跨越进程边界并被封装成 MP4。这就是为什么这个故事要从系统级优化讲起。 2. 基准测试口径与证据边界 本文把两条证据线严格分开： 两条线各自都是有效且已冻结的实验，但它们并不共用同一个源码 SHA、prompt、seed 与产物。因此我们 不 推导从基础版到 FastH3 的加速比。本文只报告 FastH3 的绝对时延，直到有一组严格对齐的 A/B 实验为止。 2.1 冻结的控制变量 两条线都从同步提交请求开始计时，到收到完整 MP4 为止。下载、启动、编译以及被排除的
Related stories
Prime Intellect launches Prime Inference serving platform for open models
Prime Intellect announced the launch of Prime Inference, a serving platform for frontier open-source models offering resilient serverless and reserved serving across multiple datacenters. The platform's production architecture includes prefill/decode optimization, NVFP4 KV compression, and NIXL transfer, as demonstrated in its GLM-5.3 deployment on GB200 NVL72.
2 reports · 1 sources
MiniMax M3.1-Flash-Preview matches Claude Opus 5.5 front-end feats
On September 28, MiniMax released M3.1-Flash-Preview, the latest M-series model with million-token context and native multimodality, available in MiniMax Code and via API subscription. Users found it can replicate front-end generation feats of Claude Opus 5.5, such as a gacha game page with full character panels, without multiple attempts.
1 reports · 1 sources
Colibrì: C-based inference framework runs 744B-param GLM-5.2 on laptops via SSD offloading, no GPU needed
Colibrì, an open-source C-based hierarchical inference framework, has gained 32k GitHub stars for enabling large MoE models to run on laptops with limited RAM by offloading inactive experts to SSD, without requiring a GPU. It supports models from GLM-5.2/5.3, DeepSeek V4 Flash, Qwen, to 975B Inkling and 2.8T Kimi K3, with RAM requirements as low as 16GB for int4-processed GLM-5.2.
1 reports · 1 sources
vLLM Adds Distribution-Preserving Gumbel-Max Text Watermarking
The vLLM project has documented its implementation of distribution-preserving Gumbel-max text watermarking. The technique uses efficient GPU kernels for watermark insertion, statistical detection for verification, and supports speculative decoding with safeguards against repeated contexts.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe