OpenDS Benchmark: Best Model Scores 39% on Real-World Deep Search
A multi-institution research team released a paper introducing OpenDS, a new task paradigm for deep search in open-web environments, along with the OpenDS-World benchmark. The benchmark includes 200 tasks covering 4 modalities, 42 resource formats, and 7 evidence access requirements; the strongest tested model achieved only 39% accuracy, highlighting the gap between current agents and real-world research capabilities.
Coverage timeline
机器之心机器之心
近年来，随着大语言模型推理和使用工具的能力不断提升，Deep Search Agent 已具备一定的自主搜索与信息整合能力。越来越多的研究开始尝试让 Agent 自主规划搜索路径，处理需要跨越多个信息源的复杂问题。 但当 Deep Search 真正进入现实互联网，研究过程会变得复杂得多。 真实网络中的信息分散在不同网站和资源中，可能藏在视频的某一帧、播客中的语段对话、学术论文的图表，或是一份需要下载才能读取的数据文件里。即使已经找到了相关网站，Agent 也可能需要继续操作交互式页面、调整查询参数，才能获取真正需要的信息。 现有 Deep Search 研究已经在多轮检索、跨来源推理、工具调用和网页交互等多方面取得了不少的进展，相关 Benchmark 也从不同角度评估了这些能力。然而， 面对多模态信息、异构资源格式与交互式访问需求交织的真实开放网络环境，当前 Agent 的端到端研究能力仍然缺乏充分评测。 这也带来了一个值得关注的问题： 当证据的位置和获取方式都不预先给定，AI Agent 能否独立判断下一步应该查什么、去哪里查、怎样访问和处理，并根据现有信息调整搜索方向，串联不同来源的证据，最终查清一个复杂问题？ 围绕这一问题，来自香港科技大学（广州）、复旦大学、华盛顿大学、帝国理工大学、斯坦福大学等多家机构的研究团队，在论文《Deep Search in Real-World Web Environments》中提出了新的深度搜索任务范式： OpenDS（Open-Web Deep Search） ，并构建了面向真实开放网络环境的评测基准 OpenDS-World 。 论文：Deep Search in Real-World Web Environments 论文链接：https://arxiv.org/pdf/2610.30181v1 GitHub 链接：https://github.com/Yu-xm/OpenDS Hugging Face 链接：https://huggingface.co/datasets/Yu2020/OpenDS-World 团队构建了 200 道任务，覆盖 4 种模态、42 种资源格式和 7 类证据访问需求，给予 GPT-6 Astra、Claude Opus 5.5、Gemini 3.8 Flash、Qwen3.8 Max、G
Related stories
DeepSeek releases V4.1-Flash-1017 open-weight model with faster, cheaper agent tasks
DeepSeek released V4.1-Flash-1017, an open-weight model checkpoint with 763B parameters, now live on its website, app, and API. The update claims stronger benchmark performance, faster task completion, and shorter outputs than its predecessor, achieved via reinforcement learning with self-feedback for end-to-end project development. DeepSeek also opened its Search & Fetch APIs to everyone, priced at $0.75 per 1000 successful requests, and requires DeepSeek Harness v0.2.1-rc.1 or later for optimal use.
9 reports · 6 sources955 points · 717 comments
OpenAI releases 722 AI-generated math manuscripts, claims on quasi-Riemann and Kakeya
OpenAI published 722 mathematical manuscripts produced by an unreleased internal frontier model, grouped into 372 result families across 17 fields, including claims on the quasi-Riemann hypothesis, BSD conjecture, and four-dimensional Kakeya conjecture. The company said most results came from a single prompt to a single AI agent, with average compute per result roughly equivalent to three hours of ChatGPT Pro thinking. Mathematicians described the release as overwhelming, noting it could take years to fully understand, and OpenAI withdrew three papers two days later.
16 reports · 6 sources1334 points · 1520 comments
DeepSeek open-sources DeepInfer inference stack, reports 82% gross margin
DeepSeek released its internal inference framework DeepInfer under the Apache 2.0 license, along with the token codec component deepseek-recipe. The framework, which supports DeepSeek-V4.1-Flash and targets Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, reportedly achieves a throughput of 155 trillion tokens per day per cluster, with API revenue of $38 per GPU-hour at off-peak pricing, implying a gross margin of 82% after GPU rental costs. The release sparked discussions about the profitability of AI inference services.
3 reports · 3 sources2335 points · 890 comments
DeepSeek global outage hits chat and API for nearly 5 hours; token compensation issued
On October 15, 2026, DeepSeek's web chat and API services experienced a global outage starting at 8:11 AM Beijing time, affecting DeepSeek V4 Pro API and V4.1 Flash API. The issue was resolved around 12:50 PM, lasting nearly five hours. As compensation, DeepSeek credited tokens to developers based on their API usage over the past 7 days, with some users reporting credits of over 300 yuan.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe