Back to News

Apple's Internalized Visual Thinking speeds proactive video reasoning 5x

#apple#video reasoning#visual chain-of-thought#internalized visual thinking

Apple researchers propose Internalized Visual Thinking (IVT), a post-training framework that internalizes predictive world modeling into model representations. During training, the model learns both text answers and latent representations of future frames, but at inference it skips generating intermediate images, directly answering from current video observations. This removes the overhead of Visual Chain-of-Thought, achieving a 5x speedup in proactive video reasoning tasks such as early-event and next-event prediction.

Coverage timeline

  1. 机器之心机器之心
    苹果新作：内化视觉思考，推理提速 5 倍

    对于能够与真实世界交互的智能系统来说，仅仅理解 “眼前发生了什么” 还远远不够。机器人、智能助手或主动视频理解系统还需要根据不完整的视觉信息推断事件如何发展，并提前预测接下来可能发生的动作。 一种直观的做法是视觉思维链（Visual Chain-of-Thought，Visual CoT）：先让模型生成一张 “想象中的未来画面”，再基于这张图像完成后续推理。未来画面能够直接表达物体位置、运动趋势、交互关系和状态变化，但生成、解码并重新编码中间图像也带来了显著开销。对于强调实时响应的主动视频推理任务而言，一个预测即使最终正确，如果在事件已经发生后才给出，价值也会大幅下降。 Apple 研究团队最近提出了一个不同的问题：如果未来视觉信息真正的价值，是帮助模型学习世界将如何变化，那么是否一定要在推理阶段把这个未来真正 “画出来”？ 围绕这一问题，团队提出 Internalized Visual Thinking（IVT，内化视觉思考），一种将预测性世界建模能力内化到模型表征中的后训练框架。IVT 在训练阶段同时学习文本答案和未来帧的隐空间表征，但在推理阶段完全移除未来视觉预测过程，只根据当前观测视频直接生成答案。 论文标题：Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning 作者单位：Apple 论文地址：https://arxiv.org/abs/2608.15869 技术博客：https://zgzxy001.github.io/blog/internalized-visual-thinking.html Visual CoT 有效，但 “把未来画出来” 太贵 这项工作关注 Proactive Video Reasoning（主动视频推理）。论文主要考虑两类任务：Early-event Prediction 要求模型在一个动作尚未结束时就识别出它；Next-event Prediction 则进一步要求模型在目标动作还没有发生前，预测下一步最可能发生什么。两者都依赖对未来的建模，但也都对推理延迟十分敏感。 论文首先比较了 Answer-Only SFT 和 Visual CoT。在 Early-event Prediction 中，Visual CoT 可以带来精度

Related stories

  • SparkDiffusion: Unified Sparse Attention, Distillation, FP8 Speeds Video Generation 265x

    Researchers from Peking University, Tsinghua University, and Alibaba propose SparkDiffusion, a unified framework integrating sparse attention, few-step distillation, and FP8 quantization to accelerate video generation. It addresses the 'high-sparsity trap' where extreme sparsity (97%) degrades output quality despite falling training loss, using staged sparse warm-up and trajectory-mixed distillation. The system achieves 265x speedup, generating a 5-second 720P video in 18 seconds, with open weights and code.

    1 reports · 1 sources

  • VBVR-Pro: 300 verifiable visual reasoning tasks, 1.25M samples, models and code released

    Researchers from NTU, CMU, and Berkeley released VBVR-Pro, a scalable and verifiable suite for native visual reasoning, including 300 tasks, 1.25 million training samples, a 100-task benchmark with verifiable scorers, and models and code. The suite supports training and evaluation of image, video, and interleaved models, and the scorers can serve as verifiable rewards in reinforcement learning.

    1 reports · 1 sources

  • Random KV Cache Eviction Matches Strong Baselines, Boosts Throughput by 43%

    Salesforce AI Research and UIUC propose Random Attention, which randomly evicts KV cache entries in reasoning traces instead of using content-based importance scores. It matches strong baselines across four models and six reasoning tasks, and improves inference throughput by 32%–43% over the strongest baseline TriAttention in vLLM serving tests.

    1 reports · 1 sources

  • Liquid AI Unveils LFM2.5-DSpark Speculative Decoding, Boosting Inference up to 3.2x

    Liquid AI announced LFM2.5-DSpark, a speculative decoding method that accelerates inference by up to 3.2x on GPUs and 2.9x on-device. The method includes day-one support for llama.cpp and SGLang, enabling faster deployment across platforms from H100 GPUs to MacBooks.

    2 reports · 2 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe