Back to News

vLLM Blog Details Encoder Disaggregation for Scaling Multimodal Serving

#vllm#encoder-disaggregation#multimodal-serving#epd

A vLLM blog post describes encoder disaggregation (EPD) for scaling multimodal serving, introducing metadata-only multimodal requests, dedicated encoder execution, remote embedding transfer, cross-encoder caching, and lower-overhead EPD routing. These techniques aim to improve efficiency and scalability in handling multimodal data.

Coverage timeline

  1. vLLM BlogInferact, InferMatrix, and IBM
    Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD

    Recent advances in vLLM encoder disaggregation: metadata-only multimodal requests, dedicated encoder execution, remote embedding transfer, cross-encoder caching, and lower-overhead EPD routing.

Related stories

  • vLLM Achieves Near-Zero-Downtime Scaling with Asynchronous Preparation and NIXL EP

    The vLLM team published a blog post describing how asynchronous preparation and NIXL EP enable near-zero-downtime scaling for Elastic Expert Parallelism (EP). The approach reduces downtime during dynamic scaling of expert-parallel models, improving operational efficiency for large-scale inference.

    1 reports · 1 sources

  • vLLM Serves DeepSeek-V4.1-Flash at 954K Tokens/s per GPU on GB300 NVL72

    vLLM's blog reports that using P/D disaggregation and Wide-EP on a GB300 NVL72 rack, it serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU, matching DeepInfer's performance. This result is 7.9x the aggregated single-node result, demonstrating significant scaling.

    1 reports · 1 sources

  • RAVEL: Training-Free Method Improves Vision Encoder Evaluation for Multimodal LLMs

    Researchers proposed RAVEL, a training-free evaluation method with two improvements—stabilized PCA whitening and patch-level similarity scoring—to predict how well vision encoders perform in multimodal LLMs. Experiments on 210 multimodal LLMs showed existing cross-modal methods degrade as candidate pools grow, while RAVEL achieved an average Spearman correlation of 0.85, outperforming prior methods by up to 101.1%.

    1 reports · 1 sources

  • NVIDIA and Waterloo Open-Source PixelUMM, an Encoder-Free Unified Image-Video Model

    NVIDIA and the University of Waterloo released PixelUMM, an encoder-free unified multimodal model that handles image and video understanding and generation directly in pixel space, bypassing ViT and VAE. It uses spatial patches for images and spatiotemporal tubelets for videos, feeding raw pixels through a linear projection into a shared Mixture-of-Transformers backbone with separate parameters for understanding and generation but shared attention.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe