CUHK-Shenzhen and Microsoft Propose ConvexLLM for Speech-Language Unified Modeling
Researchers from CUHK-Shenzhen and Microsoft Research proposed ConvexLLM, a new architecture that maps speech into the convex hull of text embeddings via a C-Gate mechanism, enabling large language models to apply their existing knowledge and reasoning to audio. In benchmarks such as VoiceBench, ConvexLLM achieved leading results among open-source models of comparable scale, offering a new path for unified speech-language modeling.
Coverage timeline
机器之心机器之心
本研究由香港中文大学(深圳)与 微软研究院(Microsoft Research)联合完成。通讯作者武执政是香港中文大学(深圳)副教授、国家级青年人才、安菲翁科技创始人。刘树杰是微软研究院首席研究员,长期从事自然语言处理、语音处理方面研究。徐明晧、王宇翔、申英达是港中深博士生、硕士生。胡宇轩和李锦宇来自微软 CoreAI。此次提出的 ConvexLLM,通过创新的 C-Gate 架构,将语音信息映射至大语言模型原有文本词向量的凸包中,使模型能够充分发挥已有的知识储备与推理能力。在 VoiceBench 等多项基准测试中,ConvexLLM 取得了同量级开源模型中的领先表现,为语音与语言的统一建模提供了新的技术路径。 论文标题:ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence 论文链接:https://arxiv.org/abs/2610.23539 作者:Ming-Hao Hsu、Shujie Liu、Yuxuan Hu、Yuxiang Wang、Yingda Shen、Jinyu Li、Zhizheng Wu 研究单位:香港中文大学(深圳)数据科学学院、微软 CoreAI、微软亚洲研究院 大模型已经学会了知识与推理。给它接上一对 “耳朵”,这些能力能不能一起接过来? 这是语音大模型绕不开的一道题:同一个问题,写出来能答,换成语音,表现却可能明显下降。声音已经被编码成向量,语言模型也足够强,两者之间还缺什么? 来自香港中文大学(深圳)与微软的研究团队,把目光放在了声音进入语言模型的接口上。ConvexLLM 把音频表示约束在语言模型原有文字词向量的凸包中,让声音以连续的文字向量组合进入大模型。 ConvexLLM Demo 演示 什么是 “凸包”?想象桌面上钉着一些钉子,用一根橡皮筋从外侧将它们围住并收紧。橡皮筋围出的区域,连同边界,就是这些点的凸包。把这些点按非负、总和为 100% 的比例做加权平均,得到的新点总会落在这个区域内。文字词向量也可以看作空间中的点,只是维度更高;ConvexLLM 就通过混合这些词向量来表示声音。 这个看起来增加了限制的设计,带来了亮眼的语音知识与推理表现。 在论文报告的同量级开源权重模型对比中,ConvexLL

Related stories
Black-Box LLM Inversion via Previous-Token Prediction Trained on Synthetic Data
A new paper presents a black-box method for reconstructing prompts from LLM outputs by training an inverse language model from scratch on synthetic data generated by the target model. Unlike prior work that fine-tunes pretrained seq2seq models on external datasets and requires model weights or logits, this approach uses previous-token prediction, mirroring forward next-token prediction, to enable faithful prompt reconstruction without auxiliary aids.
1 reports · 1 sources47 points · 10 comments
Schmidhuber-led team publishes 84-page survey on next-gen LLM architectures
A team led by Jürgen Schmidhuber, with researchers from KAUST, HKU, Huawei, and other institutions, published a comprehensive 84-page technical survey on large language model architectures, covering nearly 600 references from 1913 to 2026. The survey traces the evolution from Markov chains to Transformer and examines emerging directions such as linear attention, mixture-of-experts, multi-token prediction, and test-time training.
1 reports · 1 sources

PKU Researchers Propose MapFlow to Reuse Attention Maps Across LLMs, Cutting Prefill Latency 15-20%
Researchers at Peking University's PKU-DAIR lab propose MapFlow, a method that reuses attention maps as intermediate states across different LLMs to avoid recomputing context when switching models. In evaluations, MapFlow reduces prefill latency by 15-20% and boosts throughput by up to 25% under a P90 first-token latency constraint in multi-turn routing scenarios, while maintaining generation quality. The paper has been accepted at SIGMOD 2027 and the code is open-sourced.
1 reports · 1 sources

Multimodal Flow: Fully Continuous Architecture Unifies Language and Vision
Researchers from HUST and Horizon Robotics propose Multimodal Flow, a fully continuous multimodal architecture that models both language and vision in embedding spaces using a unified Flow Matching objective, avoiding tokenization and hybrid systems. In controlled comparisons with equal data and parameters, it outperforms hybrid and fully discrete approaches, and the 1.6B-parameter MF-1 model shows strong multimodal generalization and efficient transfer to downstream tasks.
1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe