NV-CoT: Multimodal LLMs Predict Continuous Coordinates for Visual Chain-of-Thought, NeurIPS 2026 Oral
Researchers from NTU, USTC, NUS, and Microsoft propose NV-CoT (Numerical Visual Chain-of-Thought), a method that lets multimodal LLMs directly predict continuous numerical coordinates for region selection instead of generating bounding boxes as text tokens. The work, accepted as a NeurIPS 2026 Oral (top 0.4%), supports supervised fine-tuning and reinforcement learning, addressing mismatches between token-level training and spatial error.
Coverage timeline
机器之心机器之心
一张高清照片中，远处的手提包是什么颜色？一页票据里，某项费用到底是多少？面对这些问题，多模态大模型常常需要先找到相关区域，再放大查看。但如果第一步就框错了位置，后面的推理也可能跟着出错。视觉思维链让模型能够边看边想、调用工具寻找证据。不过，许多方法仍然像生成句子一样，把边界框坐标逐个写成文本 token。 连续的空间位置，为什么要通过离散文字来表达？ 图 1 三种区域选择方式：文本坐标、固定 patch，以及 NV-CoT 的连续数值坐标。 针对这一问题，南洋理工大学、中国科学技术大学、新加坡国立大学与微软研究院的研究者提出 NV-CoT（Numerical Visual Chain-of-Thought） ：让模型直接预测连续数值坐标，将区域选择变成可学习的连续动作。该工作已获 NeurIPS 2026 Oral（Top 0.4%） ，支持监督微调与强化学习。 第一作者赵克森 是南洋理工大学博士生，主要研究多模态大模型中的强化学习。 通讯作者朱贝尔 是中国科学技术大学特任教授，主要研究多模态大模型推理、图像与视频可控生成、情感计算等。 论文标题：Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought 论文链接：https://arxiv.org/abs/2602.23959v2 项目主页：https://kesenzhao.github.io/NV-CoT/ 代码仓库：https://github.com/kesenzhao/NV-CoT Hugging Face：https://huggingface.co/kesenZhaoNTU/NVCoT-Qwen3-VL-8B-Joint-GRPO-Step32 背景与挑战 现有视觉思维链通常先输出边界框，再调用裁剪工具，将局部图像交回模型（图 1a）。然而，坐标文本化带来了两个问题。 一是 训练目标与空间误差不匹配 。真实坐标为 3.1 时，预测 3.2 比 4.1 更接近；token 级交叉熵却不会直接按照这种数值距离设置惩罚。二是数字可能被拆成多个子 token，完整的位置含义被分散到离散序列中，增加了精确生成的难度。 为避免将坐标写成文本，一些研究尝试直接利用视觉 patch 进行推理（图 1b）。例如，PaDT
Related stories
Pitt Researchers Uncover 3D Scene Topology in VLM Latent Spaces
University of Pittsburgh researchers analyzed the latent representations of vision-language models (VLMs) and found that intermediate layers encode spatial structures such as rings, grids, and random layouts, corresponding to 3D scene topology. Their paper, accepted as a NeurIPS 2026 spotlight, introduces methods to separate spatial signals from appearance attributes in high-dimensional activations.
1 reports · 1 sources
VisInteract: Interactive Text-to-Visualization for Imperfect Queries, NeurIPS 2026 Spotlight
Researchers from Hong Kong Polytechnic University and ByteDance propose VisInteract, a new paradigm for dynamic interactive text-to-visualization that handles imperfect user queries. They introduce VisInteract-Bench, a benchmark acknowledging ambiguous, incomplete, or erroneous inputs, and Vis-MCTS, a Monte Carlo tree search-based method that branches and backtracks between data retrieval, chart generation, and clarification. The work was accepted as a Spotlight paper at NeurIPS 2026 (292 out of 30,709 submissions).
Apple and Duke propose LensVLM, compressing long context into images with on-demand zoom
Apple's foundation model team and Duke University introduced LensVLM, a method that compresses long context into images and lets the model zoom into relevant pages on demand. In seven text QA benchmarks, it achieved 68.9% average accuracy with 4.3x effective compression, close to the 72.4% of full-text reference. The work was accepted at NeurIPS 2026, and a public 9B demo is available.
SAGE topological guidance mitigates long-horizon reasoning biases, NeurIPS 2026
Researchers from Virginia Tech, UW-Madison, and Dartmouth propose SAGE, a method that uses symbolic closure analysis and topological guidance to mitigate exploration and accumulation biases in long-horizon reasoning. Evaluated on 12 benchmarks across 7 model families, SAGE outperforms compared post-training methods; on Andrews–Curtis instances, Qwen3's Lean verification pass rate is nearly 8 times that of the base model. The paper is accepted at NeurIPS 2026.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe