Princeton AI Lab's SSoN benchmark: VLMs score up to 39.7%, humans 83.3%
Princeton AI Lab released the Same Space or Not (SSoN) benchmark, which tests spatial intelligence by asking models to identify the one image among four static indoor views that violates the same 3D scene. On the full 1,000-question set, humans scored 83.3% accuracy, while the best of six tested vision-language models, GPT-6 Astra Pro, reached only 39.7%, with the other five falling below the 25% random baseline. The benchmark includes strong distractor views that systematically mislead models, revealing a gap between recognizing appearance differences and understanding spatial consistency.
Coverage timeline
机器之心机器之心
AI 真的能理解三维世界吗？ 规划下一步动作、在杯子被挪走后及时调整、赶在环境变化前完成操作，动态任务不断向 AI 提出更复杂的空间要求。支撑这些行为的一块基础，是对眼前空间的可靠理解：物体在哪里，不同视角的观察能否彼此一致？ 普林斯顿 AI Lab 的研究者们带来了一张空间智能新考卷：Same Space or Not（SSoN）。 它拿掉了实时变化和动作控制，只给四张静态室内图像，让模型找出其中违背同一场景的那一张，并解释为什么。 项目主页：https://sson.aisexam.com/ 论文链接：https://arxiv.org/abs/2610.23959 数据集：https://huggingface.co/datasets/HuiziCao/SSoN 代码链接：https://github.com/sson-benchmark/SSoN 微调权重：https://huggingface.co/HuiziCao/SSoN-Qwen2.5-VL-7B-LoRA 结果却拉开了明显差距：在完整 1,000 道题上，人类准确率达到 83.3%。六个受测视觉语言模型（VLM）中，GPT-6 Astra Pro 以 39.7% 领先，其余五个全部低于四选一的 25% 随机基线。没有动态交互的复杂性，人机差距已经出现在更基础的静态跨视角理解上。 为什么会低于 25%？这是等概率盲猜的期望，并非能力下限。SSoN 专门构建了 “看着像变了、空间上却正确” 的强干扰视图。模型若能发现外观差异，却无法判断它是否来自视角变化，就可能被系统性带偏，表现反而低于盲猜。这不等于毫无视觉能力，而是暴露出从识别差异到理解空间之间的缺口。 而 39.7%，还只是选对答案的成绩，其中一些回答的理由仍然站不住脚。 为什么 17 个视觉编码器全部低于随机线？让模型实际执行三维重建，为什么仍没答对一半？这张考卷把 “空间智能卡在哪里”，一层层拆了出来。 图 1｜SSoN 任务与完整 1,000 题上的成绩。人类答案准确率 83.3%，受测 VLM 最高 39.7%；柱状图进一步区分有充分证据支持的正确选择与解释不成立的正确选择。 空间智能的集体偏科： 交换和细节相对容易，旋转和移动却成共同盲区 SSoN 的规则很简单：四张图中，三张来自未改变的三维场景，一张来自编辑后的版本。没有指定的 “正
Related stories
Anthropic survey: 91% of staffers vs 39% of US adults expect AI Nobel-level discovery by 2030
Anthropic published a survey comparing US adults' and its own staffers' expectations for AI capabilities. The survey found that 91% of Anthropic staffers expect AI to make a Nobel-level discovery by 2030, compared to 39% of US adults. US adults expect rapid AI advancement, but Anthropic staff expect even faster progress.
1 reports · 1 sources
Artificial Analysis launches Intelligence Index v5 with private coding and science benchmarks; Claude Opus 5.5 leads at 58
Artificial Analysis released the Intelligence Index v5, adding two new benchmarks and three upgrades, with 60% of the index now based on private test sets or held-out answers. Claude Opus 5.5 (max) leads the composite measure with a score of 58, reflecting model capabilities across real-world tasks.
SLDBench v2: Frontier Models Degrade When Planning Scaling-Law Experiments
Researchers from CUHK-Shenzhen, HKU, Tsinghua, and PKU released SLDBench v2, an ICLR 2026 benchmark where AI must autonomously choose scaling-law experiments from a priced catalog with real compute costs. All 8 frontier models degraded when selecting experiments, with prediction errors increasing 1.5 to 4.7 times. Best performer Claude Opus 5.5 used only 4% of full data cost yet surpassed classical power-law fitting accuracy.
1 reports · 1 sources
OpenDS Benchmark: Best Model Scores 39% on Real-World Deep Search
A multi-institution research team released a paper introducing OpenDS, a new task paradigm for deep search in open-web environments, along with the OpenDS-World benchmark. The benchmark includes 200 tasks covering 4 modalities, 42 resource formats, and 7 evidence access requirements; the strongest tested model achieved only 39% accuracy, highlighting the gap between current agents and real-world research capabilities.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe