Stanford, Together AI propose Next-Concept Modeling for parallel text generation, 10-31x speedup
Researchers from Stanford University and Together AI introduced Next-Concept Modeling (NCM), a coarse-to-fine parallel text generation paradigm adapted from next-scale prediction used in image generation. In experiments, NCM achieved 10-31x speedup over diffusion language models with comparable or better quality, offering an alternative to both autoregressive and diffusion approaches.
Coverage timeline
机器之心机器之心
编辑｜泽南 如今的大模型再聪明，输出内容也是一个字一个字往外蹦。有没有一种可能：真正的 AGI 不是这样？ 近日，斯坦福大学与 Together AI 的研究者提出了一种既不同于自回归，也不同于扩散的第三种文本生成范式。他们把 NeurIPS 2024 最佳论文 VAR 在图像上的「下一尺度预测」配方搬进文本，让模型先并行写出几句话的「概念大纲」，再逐层并行填充细节，直到还原成完整文字。 于是新模型就叫 Next-Concept Modeling （NCM，下一概念建模），对主流 Next-Token Modeling（下一 token 建模） 来了个正面叫板。 在若干实验中， NCM 以相近或更好的质量，实现了相对扩散语言模型 10—31 倍的生成速度 。 论文：《Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation》 论文链接：https://arxiv.org/abs/2610.14162 GitHub：https://github.com/HazyResearch/ncm 我们知道，目前主流的大语言模型即自回归模型（包括不限于 GPT、Claude、DeepSeek），通常根据已有内容预测下一个 token，后续输出依赖前面的结果。近两年逐渐热门的扩散语言模型则可以同时处理多个位置，通过反复去噪逐步生成文本。但并行不意味着只需一步：迭代次数减少则质量可能下降，每轮处理整段序列也会带来更多开销。 NCM 的思路是连同「每一步到底生成什么」一起改变。它借鉴了图像生成中的 next-scale prediction，画面可以先生成低分辨率版本，再逐级增加细节。既然文本可以扩散生成，文本是否也能拥有自己的「缩略图」呢？ 图片来自 NeuralPS 2024 Best Paper《Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction》。 难点在于这是两个维度的事：图片缩小后通常还能看出主体，一句话删掉几个字就可能完全变了意思。因此，我们需要让模型自己学会一种既概括、又适合继续细化的表示。NCM 为此设计了两个训练阶段。 首先是 训练一个多尺度残差 VQ-VAE 。我们可以把
Related stories
Apple and CMU propose SSR framework cutting multimodal search agent inference latency by over 90%
Researchers from Apple and Carnegie Mellon University propose SSR (Selection-based Structured Reasoning), a framework that replaces free-form token-by-token reasoning with selection from predefined reasoning options in multimodal search agents. In experiments across 7 multimodal search benchmarks, SSR reduces per-step inference latency by over 90% while maintaining task success rates comparable to free-form thinking.
1 reports · 1 sources
Tadpole: Autoencoder Foundation Model for 3D PDEs with Online Learning
Researchers propose Tadpole, an autoencoder-based foundation model for 3D partial differential equations (PDEs), which learns to compress physical states before predicting evolution, with online learning capabilities. The approach differs from typical predictive models by first learning the spatial structure of physical fields, aiming to improve transferability across different physical systems.
Tsinghua, Infinigence-AI, SJTU Open-Source On-Device Inference Engine APXInf for Embodied Models
On September 15, Tsinghua University, Infinigence-AI, and Shanghai Jiao Tong University open-sourced APXInf, an on-device inference engine for embodied models. The engine reportedly achieves a 10.7x end-to-end speedup on robot hardware, reducing inference latency from 278ms to 26ms on Thor chips with FP8 configuration, enabling real-time control at 38.46Hz.
SparkDiffusion: Unified Sparse Attention, Distillation, FP8 Speeds Video Generation 265x
Researchers from Peking University, Tsinghua University, and Alibaba propose SparkDiffusion, a unified framework integrating sparse attention, few-step distillation, and FP8 quantization to accelerate video generation. It addresses the 'high-sparsity trap' where extreme sparsity (97%) degrades output quality despite falling training loss, using staged sparse warm-up and trajectory-mixed distillation. The system achieves 265x speedup, generating a 5-second 720P video in 18 seconds, with open weights and code.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe