Apple and CMU propose SSR framework cutting multimodal search agent inference latency by over 90%
Researchers from Apple and Carnegie Mellon University propose SSR (Selection-based Structured Reasoning), a framework that replaces free-form token-by-token reasoning with selection from predefined reasoning options in multimodal search agents. In experiments across 7 multimodal search benchmarks, SSR reduces per-step inference latency by over 90% while maintaining task success rates comparable to free-form thinking.
Coverage timeline
机器之心机器之心
在智能体的典型工作流程中，模型每执行一个动作之前，都要先生成一段自由形式推理（free-form thinking）：分析当前观察、评估已有证据、再决定下一步行动。这一 “思考” 过程是 ReAct 范式的核心环节，但也带来了实打实的推理开销。 来自苹果与卡内基梅隆大学的研究团队近期提出了一种新框架 SSR（Selection-based Structured Reasoning），对这一环节进行了重新设计：与其逐 token 生成推理，不如让模型从一组预定义的推理选项中 "做选择题"。实验结果显示，这一方法在 7 个多模态搜索 benchmark 上将每轮推理延迟降低超过 90%，同时保持了与 free-form thinking 相当的任务成功率。 论文标题：Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents 作者单位：Apple，Carnegie Mellon University 论文地址：https://arxiv.org/abs/2610.01892 项目主页：https://zfy0314.github.io/ssr-webpage/ 一、动机：小模型的 "废话文学" 值得吗？ 如今，多数搜索 Agent 都遵循类似 ReAct 的交互范式：观察（Observation）→ 推理（Reasoning）→ 动作（Action），如此循环往复，直到任务完成。 以一道多模态搜索题为例：模型看到一张建筑的照片，问题是 "这栋建筑里接下来会举办什么展览"。一个典型的 Agent 会先生成一段 free-form thinking："这张图片看起来像某个博物馆…… 我需要先判断建筑是什么…… 然后搜索它最近的展览……"，随后据此构造搜索 query 并调用工具。 然而，这种 "先想后做" 的模式在小模型上暴露出明显的性价比问题。研究人员观察到，2B、4B 这类端侧小模型有时会生成相当长的推理内容，但这些内容对生成下一步动作和最终决策却未必有帮助。 更关键的是，研究团队通过分析大量搜索轨迹发现：尽管每道题的实体、查询和观察结果千差万别，但决定下一步动作的高层推理逻辑却是反复出现的。比如 “某场 AI 会议在哪里举办” 和 “LEGO 是哪年
Related stories
Stanford, Together AI propose Next-Concept Modeling for parallel text generation, 10-31x speedup
Researchers from Stanford University and Together AI introduced Next-Concept Modeling (NCM), a coarse-to-fine parallel text generation paradigm adapted from next-scale prediction used in image generation. In experiments, NCM achieved 10-31x speedup over diffusion language models with comparable or better quality, offering an alternative to both autoregressive and diffusion approaches.
1 reports · 1 sources
Tsinghua, Infinigence-AI, SJTU Open-Source On-Device Inference Engine APXInf for Embodied Models
On September 15, Tsinghua University, Infinigence-AI, and Shanghai Jiao Tong University open-sourced APXInf, an on-device inference engine for embodied models. The engine reportedly achieves a 10.7x end-to-end speedup on robot hardware, reducing inference latency from 278ms to 26ms on Thor chips with FP8 configuration, enabling real-time control at 38.46Hz.
Tsinghua introduces SRHarness and MechBench for agentic symbolic regression and mechanism discovery
Tsinghua University researchers introduced SRHarness, a runtime framework for agentic symbolic regression, and MechBench, a benchmark for mechanism discovery, aiming to improve AI's ability to derive formulas from data and understand underlying physical mechanisms. Experiments show SRHarness achieves 72.97% symbol recovery with DeepSeek-v4-flash-0731, far exceeding Codex's 20.72%, while MechBench reveals that even GPT-5.6-sol-powered Codex recovers mechanisms at only 13.75% on the core set, highlighting a significant gap between pattern discovery and mechanism understanding.
Random KV Cache Eviction Matches Strong Baselines, Boosts Throughput by 43%
Salesforce AI Research and UIUC propose Random Attention, which randomly evicts KV cache entries in reasoning traces instead of using content-based importance scores. It matches strong baselines across four models and six reasoning tasks, and improves inference throughput by 32%–43% over the strongest baseline TriAttention in vLLM serving tests.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe