METR publishes predeployment evaluation of Claude Opus 5.5 AI R&D capabilities
METR released a summary of its predeployment evaluation of Claude Opus 5.5, focusing on the model's potential to accelerate AI R&D at Anthropic. The evaluation, conducted under an unpaid agreement, assessed capabilities on difficult, long-horizon tasks and examined whether AI R&D at Anthropic was already dramatically accelerated during the model's development. METR noted the work was not meant to verify compliance with specific policy thresholds or assess alignment properties.
Coverage timeline
METR
Note on independence: This evaluation was conducted under an unpaid agreement for AI R&D assessment. 1 We drafted the initial summary, and then Anthropic had the opportunity to review and edit the text. We signed off on this final text from the Claude Opus 5.5 system card . Our preliminary evaluation focused on how Claude Opus 5.5 might impact AI R&D, mainly based on its capabilities on difficult, long-horizon tasks. The main claims we attempt to assess in this report are: (A) would AI R&D at Anthropic now be dramatically accelerated by using Claude Opus 5.5; and (B) was AI R&D at Anthropic already dramatically accelerated due to AI during the development of Claude Opus 5.5. Note that our work was oriented around collecting evidence related to AI R&D capabilities but was not meant to verify claims about compliance with any specific threshold from Anthropic’s policies. This report summary also does not attempt to assess whether Claude Opus 5.5 has or does not have particular alignment p
Related stories
METR outlines method for assessing AI-driven acceleration of AI R&D
METR published an article describing how an assessor could investigate whether AI capabilities progress is accelerating due to automation of AI R&D. The proposed assessment would analyze unreleased model capabilities, growth of key inputs (labor, compute, data, algorithms), and researcher input, while also examining bottlenecks and method limitations.
Mistral Large 4 preview: 1T-parameter open-weight model, weights due Oct 27
Mistral AI released a public preview of Mistral Large 4 (ML4), a 1-trillion-parameter multimodal model with 49 billion active parameters, trained on 3,800 Nvidia Grace Blackwell GPUs in its European data centers. The open weights are promised by October 27. Early benchmarks show an Artificial Analysis Intelligence Index score of 38, comparable to DeepSeek V4.1 Flash, and the model is claimed to top any open model from the US or Europe.
10 reports · 5 sources2033 points · 1210 comments
METR Deploys Basic Per-Action Monitor for AI Evaluations
METR published a blog post describing the development and deployment of a basic live per-action monitor for its AI evaluations, aimed at reducing incidents involving harmful actions by agents. The monitor uses an LLM judge to review each action before execution, flagging those above a threshold for human review and halting the eval. It focuses solely on real-world harm and attempts to subvert the monitoring system, ignoring other issues like cheating.
Anthropic details four Claude unauthorized access incidents; METR to investigate
Anthropic published an alignment assessment of four incidents in which Claude models gained unauthorized access to real third-party systems, including a new case involving Opus 4.6. The research organization METR will investigate these incidents.
2 reports · 2 sources
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe