SenseTime-led study: GPT-6-Astra scores near-perfect on benchmarks but fails real-robot tasks
A research report by SenseTime, Daxia Robotics, Ropedia, NTU Singapore, and the University of Tokyo analyzes GPT-6-Astra's embodied intelligence across 12 simulation benchmarks and 8 real-robot platforms. The model achieves near-perfect scores on traditional benchmarks (e.g., 100% on LIBERO-Long++, 99.3% on LIBERO-Pro) but fails on real-robot tasks, such as taking over 4 hours and $55 to place a pen in a holder, and failing 11 insertion attempts. The report concludes GPT-6-Astra has largely crossed the embodied intelligence threshold but remains far from practical deployment.
Coverage timeline
机器之心机器之心
编辑｜杜伟 上月发布 GPT-6-Astra 时，OpenAI 称它为「全球智能程度最高且最符合人类意图的模型」。具身社区则是惊讶的发现，VLA vs WAM 的路线之争，被通用大模型用文字模态「降维打击」了，大家不禁想问： 「GPT-6-Astra 的具身能力上限到底在哪？具身社区下一步最值得做的是什么？」 近日，商汤联合大晓机器人、Ropedia、新加坡南洋理工大学和东京大学的研究团队在《Has GPT-6-Astra Achieved Embodied Intelligence?》中，给出了 覆盖 12 项仿真评测、8 个真实机器人平台的系统分析 。 他们还分析了 GPT-6-Astra 与底层机器人控制系统的协作方式，搭建了一套更贴近真实家庭需求的具身评测原型。 技术报告地址：https://www.sensenova.ai/has-gpt-6-astra-achieved-embodied-intelligence 有意思的是，同一个 GPT-6-Astra，在简单解除限制之后，在多项传统具身评测中交出了满分或接近满分的成绩：其中在长程组合操作评测基准 LIBERO-Long++ 上达到 100%，在考察操作鲁棒性和泛化能力的 LIBERO-Pro 上达到 99.3%，人形机器人基准 SIMPLE 上达到 98.8%，RoboDojo 上也可以达到 81.9%。但当这个模型指挥起真实机器人干活，一些日常小事，依然让它犯了难。 「把笔放进笔筒这个对人来说几秒钟就能完成的任务，总耗时 4 个多小时，消耗 55 美元。」 「明明判断鸡蛋煎好了，却没能及时关火，最后把鸡蛋烧焦；打开冰箱放下牛奶后，又转头忙着搬水果，让冰箱门敞开了好几分钟。」 「插入插头的操作，在多款真机交叉验证 11 次，一次都没成功。」 高分与失误放在一起，明显的反差暴露出一个关键问题：有能力规划、观察并纠正动作的通用大模型，距离可靠地干活，还有多远？ 报告给出的判断是： GPT-6-Astra 已经在很大程度上跨过了具身能力的门槛，但距离能够实用部署的具身智能仍有相当差距 。实验中暴露出的成本及耗时的问题、那些意想不到的失误，恰好说明接下来的最值得啃的硬骨头在哪里。 值得注意的是，在去年 GPT-5 发布后，相同团队也做过一篇针对其空间智能的评测文章《Has GPT-5 Achieved Sp
Related stories
LatePost Embodied-AI Quarterly: GPT-6 Astra Raises Question of Whether General Models Will Absorb Robotics
LatePost's Q3 2026 embodied-AI quarterly report examines whether general multimodal foundation models like OpenAI's GPT-6 Astra will absorb embodied intelligence, noting that Astra can directly output robot end-effector poses and outperform some embodied models on certain tasks. The report also covers AMD's planned $8.2 billion acquisition of World Labs, Unitree's IPO, and FCC restrictions on new Chinese robot imports, prompting a rethinking of the boundary between physical AI and general AI.
1 reports · 1 sources
Redwood Research: Filler tokens boost GPT-6-Astra reasoning, complicating monitoring
Redwood Research reports that GPT-6-Astra performs significantly better on serial reasoning tasks when prompts are padded with meaningless filler tokens, improving from ~10% to ~50% on 4-hop natural facts reasoning and from ~60% to ~90% on old AIME problems. The finding raises interpretability concerns because the model can perform substantial cognition without verbalizing it in chain-of-thought, making monitoring harder.
1 reports · 1 sources
UniPat AI's PaperBenchX: GPT-6 Astra full reproduction rate only 13.98% on 93 tasks
UniPat AI's PaperBenchX benchmark evaluated 10 model configurations on 93 real paper replication tasks, finding that the strongest model, GPT-6 Astra, achieved only a 13.98% full reproduction rate. This comes amid debate over AI math benchmarks, with 25 Fields Medalists including Tao and Deng Yu signing an open letter warning that treating math problem-solving as a benchmark is harmful to mathematics and the community.
1 reports · 1 sources
Artificial Analysis: Gemini 4 Argon matches GPT-6 Astra on intelligence, cuts hallucination to 15%
Artificial Analysis reported that Google's Gemini 4 Argon (high) matches GPT-6 Astra (max) on its Intelligence Index, with a 15% hallucination rate versus 51% for Astra. The model also costs 60% of Astra's per-task price under discounted pricing.
1 reports · 1 sources
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe