Back to News

vLLM Adds Day-0 Support for Thinking Machines' Inkling 2 Small

#vllm#inference#speculative-decoding#model-support

vLLM announced day-0 support for Thinking Machines' Inkling 2 Small model. The integration includes attention residuals, a new decode kernel for relative-bias attention, all-reduce fused with surrounding operations, and DSpark speculative decoding.

Coverage timeline

  1. vLLM BlogInferact and Thinking Machines
    Inkling 2 Small on vLLM: Day-0 Support

    vLLM supports Thinking Machines' Inkling 2 Small from day 0: attention residuals, a new decode kernel for relative-bias attention, all-reduce fused with the ops around it, and DSpark speculative decoding.

Related stories

  • vLLM Adds Day-0 Support for Flint Sparse Model Checkpoints

    vLLM announced day-0 support for Flint checkpoints, a sparse model with 40 billion total parameters and 2.5 billion active parameters. On a single GPU, vLLM decodes a single request at approximately 500 tokens per second using these checkpoints.

    1 reports · 1 sources

  • vLLM Adds Day-0 Support for Kimi K3.1 Pro with Sparse Attention and Packed KV Cache

    vLLM announced day-0 support for Kimi K3.1 Pro, enabling sparse attention on a KDA hybrid, a packed KV cache with FP8 or NVFP4 latents and MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode. This integration allows users to run the model immediately upon release with optimized performance.

    1 reports · 1 sources

  • Thinking Machines Lab debuts Inkling 2 Small, 594B-param MoE; weights due next week

    Thinking Machines Lab introduced Inkling 2 Small, a mixture-of-experts model with 594B total and 17B active parameters, initially available as a proprietary model via its Tinker platform and Tinker Playground. The company plans to release the weights next week after safety testing. Artificial Analysis reports the model matches Gemini 3.8 Flash (high) in its index and would be the top open model outside China once weights are released.

    2 reports · 1 sources

  • vLLM Adds Day-Zero Support for Mistral Large 4 on ROCm with TP8/TP4

    vLLM announced day-zero support for serving Mistral Large 4 (mistralai/Mistral-Large-4-1T-A52B) on AMD ROCm, with configurations for tensor-parallel size 8 (one replica per node) and tensor-parallel size 4 (two replicas per node). The setup uses AITER optimizations, FP8 MoE requantization (mxfp4), speculative decoding with dspark, and tool-call/reasoning parsers.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe