Fudan, Tencent Hunyuan, Tsinghua Release ExplorationBench to Measure AI Rule Discovery
Fudan University, Tencent Hunyuan, and Tsinghua University jointly released ExplorationBench, a benchmark designed to evaluate how AI systems discover rules in unfamiliar environments. It introduces two alien worlds, AlienCode and AlienLogic, where standard rules are altered, and tests 10 frontier AI systems through autonomous exploration. Initial results show that without environmental feedback, performance caps at 11.0%, while with exploration, the best trajectory reaches 89.0%.
Coverage timeline
机器之心机器之心
近日，腾讯混元官方账号（@TencentHunyuan）在 X 上发布了一项新研究：ExplorationBench，一个专门衡量 AI 系统如何「探索」的评测基准；随后，腾讯 AI 官方账号（@TencentAI_News）也发文进行了专题推介。 这项工作由此前推出 CL-bench 的复旦大学与腾讯混元团队，联合清华大学共同完成。腾讯混元在推文中这样概括它和 CL-bench 的关系：「CL-bench 问的是模型能否从上下文中学习，ExplorationBench 问的是模型能否自己发现规则。」 腾讯 AI 官方账号（@TencentAI_News）在推介中则把 ExplorationBench 称为一场「AI 无法提前备考的测试」（the exam AI can't study for）：「大多数基准测试的问题背后都有几十年的公开讨论，所有这些文本都已在训练数据里，因此高分可能只是意味着好记性。我们造了两个模型过往所学完全失效的异星世界……榜单上的每一个分数，测量的都是真正的发现能力。」 过去几年，大模型的成绩单越来越漂亮：奥数难题、复杂编程、专业资格考试，前沿模型几乎都能拿下。但这些成绩单测的，大多是模型已经知道什么。换一个问法，会更接近真实世界：如果把模型放进一个它完全没见过的环境，规则陌生，手里只有一份写错了的说明书，它能不能像人一样边试边学，自己把规律摸清楚？ 为此，研究团队搭了两个规则被悄悄改过的「异星世界」：编程语言 AlienCode 和逻辑证明系统 AlienLogic。在 AlienCode 里，SHATTER 做的是乘法，EMIT(100) 打印出 127，整门语言里还没有加法。Claude Opus 5、GPT-5.6 Sol、Gemini 3.8 Flash、Grok 4.6、Kimi K3、Qwen3.8-Max、Hy4 preview 等 10 个前沿 AI 系统被放进去，只能靠自己写程序、做证明、看反馈，一点点摸索规则。 结果相当有意思。读完示例时，AlienCode 上没有一条轨迹超过 15.7%；四轮探索后，最好的轨迹达到 89.0%；可如果只让模型多想、不给环境反馈，成绩最高只有 11.0%。更反直觉的是，在 AlienCode 里，GPT-5.6 Sol 靠自主探索拿到 88.1%，探索前直接把完整规则告诉它，反而只有
Related stories
Tsinghua introduces SRHarness and MechBench for agentic symbolic regression and mechanism discovery
Tsinghua University researchers introduced SRHarness, a runtime framework for agentic symbolic regression, and MechBench, a benchmark for mechanism discovery, aiming to improve AI's ability to derive formulas from data and understand underlying physical mechanisms. Experiments show SRHarness achieves 72.97% symbol recovery with DeepSeek-v4-flash-0731, far exceeding Codex's 20.72%, while MechBench reveals that even GPT-5.6-sol-powered Codex recovers mechanisms at only 13.75% on the core set, highlighting a significant gap between pattern discovery and mechanism understanding.
1 reports · 1 sources
Zhizhi Innovation open-sources IQuest-Q1, 320B-param sparse MoE coding model
Chinese research institute Zhizhi Innovation released IQuest-Q1, a sparse MoE model with ~320B total parameters and ~15B active parameters, focused on coding, software engineering, and complex agentic tasks. The model shows balanced performance across benchmarks including NL2Repo, CyberGym, Terminal-Bench 2.1, DeepSWE v1.1, and JobBench, with weights and blog published on GitHub and Hugging Face.
Tsinghua, Infinigence-AI, SJTU Open-Source On-Device Inference Engine APXInf for Embodied Models
On September 15, Tsinghua University, Infinigence-AI, and Shanghai Jiao Tong University open-sourced APXInf, an on-device inference engine for embodied models. The engine reportedly achieves a 10.7x end-to-end speedup on robot hardware, reducing inference latency from 278ms to 26ms on Thor chips with FP8 configuration, enabling real-time control at 38.46Hz.
BenchShield Framework Detects Reward Hacking in LLM-Agent Evaluations
Researchers from Dartmouth, UC Berkeley, and BenchFlow AI introduced BenchShield, a framework for detecting reward hacking in LLM-agent evaluation infrastructure. It extends checks to the entire scoring process, using static analysis to find exploit paths and runtime evidence to determine if agents attempted or used them. The paper highlights a case where a Lean proof verifier gave full marks to an incorrect proof due to disabled kernel type checking.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe