DSpine parallel speculative decoding achieves up to 4x SGLang throughput, open-sourced
Researchers from Tencent Youtu Lab, Shanghai Jiao Tong University, and Xiamen University released DSpine, a parallel speculative decoding method that injects forming prediction features from predecessor positions into successors at each layer of the draft network, then uses a transition cache for conditional correction. The method achieves up to 4x SGLang throughput and is open-sourced, addressing the suffix decay problem in parallel drafting.
Coverage timeline
机器之心机器之心
在大型语言模型（LLM）的无损推理加速中，推测解码（Speculative Decoding）已成为工业界标配。以 EAGLE-3 为代表的自回归草稿模型虽然预测准，但每多猜一个 token 就要多跑一步串行前向；而受扩散语言模型启发的 DFlash 将草稿生成彻底并行化 —— 一次骨干网络前向即可同时吐出整块候选 token，大幅压缩了起草耗时。 然而，并行起草始终面临一个核心悖论： 后一个位置在计算时，根本不知道前一个位置最终会选出哪个词。 这种 “盲猜” 会导致严重的多模态碰撞与后缀衰减（Suffix Decay）：每个位置单独看概率都合理，拼在一起却前后矛盾。由于目标大模型按从左到右的严格前缀顺序进行验证，一旦中间某个位置衔接失败被拒，后面算好的所有草稿 token 都会全部作废。 为了补齐并行草稿的因果依赖，近期 DeepSeek 提出的 DSpark 以及 Domino、DFlash2 等工作纷纷给出了改进方案。但现有方法要么把依赖注入留到骨干网络计算完之后（如 Domino、DSpark 的轻量串行头），导致后继位置在特征形成阶段无法感知前驱倾向；要么在骨干内部直接混合相邻隐状态（如 DFlash2 的局部卷积），将上下文信息与预测信号混为一谈。 针对这一瓶颈，来自 腾讯优图实验室 、 上海交通大学 和 厦门大学 的研究团队提出了 DSpine（Draft in Parallel, Condition Through Depth） 。 DSpine 的核心思想可以用八个字概括： 同时起草，逐层接力 。它不再苦等前一个位置最终选定离散 token，而是从草稿网络的第 1 层开始，就将前驱位置 “正在形成中的预测特征” 提取到统一的词元嵌入空间，逐层注入给紧邻后继；到了网络末端选词时，再通过转移缓存（Transition Cache）并行预计算相邻候选组合分数，用已确定的前驱 token 完成条件修正。 论文标题：Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding 论文链接：https://arxiv.org/abs/2609.36173 开源代码：https://github.com/tencent/dspine 模型
Related stories
HKUST and Huawei propose BalanceRoute to cut MoE LLM serving load-balancing bottleneck
Researchers from HKUST and Huawei's 2012 Lab propose BalanceRoute, an online routing method that models synchronous waiting in MoE LLM serving and converts short-term load prediction into routing decisions. In a 96-GPU test, the predictor-based variant BR-H improves decode throughput by about 15% over the strongest baseline. The work is accepted by NeurIPS 2026 and deployed in the jointly built MaaS system MathBot-Pie.
1 reports · 1 sources
DeepSeek open-sources DeepInfer inference stack, reports 82% gross margin
DeepSeek released its internal inference framework DeepInfer under the Apache 2.0 license, along with the token codec component deepseek-recipe. The framework, which supports DeepSeek-V4.1-Flash and targets Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, reportedly achieves a throughput of 155 trillion tokens per day per cluster, with API revenue of $38 per GPU-hour at off-peak pricing, implying a gross margin of 82% after GPU rental costs. The release sparked discussions about the profitability of AI inference services.
Stanford, Together AI propose Next-Concept Modeling for parallel text generation, 10-31x speedup
Researchers from Stanford University and Together AI introduced Next-Concept Modeling (NCM), a coarse-to-fine parallel text generation paradigm adapted from next-scale prediction used in image generation. In experiments, NCM achieved 10-31x speedup over diffusion language models with comparable or better quality, offering an alternative to both autoregressive and diffusion approaches.
Nine Chapter Cloud's Alaya-DFlash2 speeds up GLM-5.3 decoding by 3.41x
Nine Chapter Cloud introduced Alaya-DFlash2, a speculative decoding solution for GLM-5.3 that combines a parallel draft model with a path selection mechanism to maintain high acceptance length in long contexts. Benchmarks on 8 H200 GPUs show a 3.41x single-user decoding speedup (288.2 vs 84.5 tokens/s) and up to 2.24x total output throughput at 4 concurrent requests, with a 44.9% throughput gain at 64 concurrency.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe