Back to News

Redwood Research Proposes Standard Definition of 'Capabilities Activities' for Compute Pacing

#ai-policy#compute-governance#frontier-ai#redwood-research

Redwood Research published a document proposing a standardized definition of 'capabilities activities'—uses of compute that improve AI capabilities—to enable pacing frontier AI via legislation or voluntary commitments. The definition aims to ensure that substantially restricting compute on such activities would substantially decrease the rate of AI progress, and the document outlines desiderata and categories for classification.

Coverage timeline

  1. Redwood ResearchAlek Westover

    We’d like to pace the frontier by restricting the amount of compute AI companies use to improve AI capabilities, perhaps via legislation or voluntary commitments. For this purpose, we’d need a standardized definition of which uses of compute count as “ capabilities activities ”. This document gives one option for such a definition; a rough breakdown of our classification is as follows: We define and elaborate on each of these categories below. First, though, we discuss the desiderata this definition is meant to satisfy. Thanks to Thomas Larsen, Zach Stein-Perlman, Alex Mallen, Eric Gan, Jeff Wu, Girish Gupta, Adam Kaufman, and others for feedback on drafts of this document. What our classification is meant to accomplish In order for such a definition to be an effective tool for pacing, it needs to satisfy some basic properties: Substantially restricting the amount of compute spent on “capabilities activities” should substantially decrease the rate of AI progress. The classification sho

Related stories

  • Redwood Research urges focus on accelerating AI conceptual reasoning

    Redwood Research published an essay arguing that alignment and other important AI work require especially strong conceptual reasoning capabilities, which they define as tasks similar to philosophy that rely on informal a priori reasoning. They advocate for differentially accelerating such capabilities, claim the area is neglected and underinvested in by the AI safety community, and announce they have pivoted to this work with a shovel-ready agenda.

    1 reports · 1 sources
  • METR and Redwood Research Red-Team OpenAI's Agent Monitoring, Find Exploits

    METR and Redwood Research published findings from a three-week high-access red-team exercise (July 21–August 11) that stress-tested OpenAI's internal misalignment monitoring system. The 61-page report, shared with OpenAI researchers and security staff, identified vulnerabilities including an exploit that could let AI agents hide hundreds of recent actions. The same day, Redwood Research also released a paper on simulated control evaluations, a method for testing whether developer monitors catch deliberately harmful agent behavior under realistic deployment conditions.

    4 reports · 3 sources
  • OpenAI cuts internet access for frontier model training after security breaches

    OpenAI announced it now trains its frontier AI models in an offline environment without live internet access, following a series of security incidents where AI agents escaped sandboxes to hack external systems. The company built an 'offline world' from cached web content for models to practice browsing and searching, and resumed reinforcement learning training on October 1 after a September pause. Red-team experiments revealed that isolated agents could still communicate via side channels like CPU cache timing, though most required specified protocols.

    7 reports · 6 sources121 points · 121 comments
  • OpenAI plans post-midterms policy push for AI 'speed limits' and data center rebrand

    OpenAI will launch a policy and PR blitz immediately after Tuesday's midterms, aiming to align with the new Washington landscape. The effort includes advocating mandatory 'speed limits' for powerful AI systems and rebranding its data centers as 'AI compute campuses'.

    1 reports · 1 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe