Redwood Research urges focus on accelerating AI conceptual reasoning
Redwood Research published an essay arguing that alignment and other important AI work require especially strong conceptual reasoning capabilities, which they define as tasks similar to philosophy that rely on informal a priori reasoning. They advocate for differentially accelerating such capabilities, claim the area is neglected and underinvested in by the AI safety community, and announce they have pivoted to this work with a shovel-ready agenda.
Coverage timeline
Redwood ResearchEmery Cooper
Introduction In this post, we make the case that alignment and other important work require especially strong conceptual reasoning capabilities. By this, we roughly mean tasks that are similar to philosophy in that they aren’t straightforwardly verifiable, and progress relies on informal a priori reasoning and argumentation. We argue that differentially accelerating AI’s conceptual reasoning capabilities is important. We also think it’s neglected, and that the AI safety community currently underinvests in this area. We have ourselves pivoted from doing conceptual research directly to working mainly on differentially accelerating AI’s conceptual reasoning. We have a shovel-ready agenda which can absorb more people. In a follow-up post, we will lay out at a high level how to work on this problem. If you are interested in doing this work, get in touch! Thanks for reading Redwood Research blog! Subscribe for free to receive new posts and support my work. For others discussing related worri
Related stories
Redwood Research tests distillation for AI safety incrimination and capabilities
Redwood Research released a paper empirically testing two distillation approaches for AI safety: Distillation for Incrimination (DFI) and Distillation for Capabilities (DFC). DFI distills an untrusted model into a weaker trusted one to transfer misalignment without the ability to hide it, and tests on AuditBench secret-keeping organisms show students admit hidden quirks more often than teachers. DFI works best when the student shares the teacher's pretrained base, suggesting targeting the teacher's pre-RL checkpoint.
Redwood Research: Capabilities Research Expands Safety-Usefulness Pareto Frontier
Redwood Research published an essay arguing that a common definition of safety research—research that improves safety without reducing usefulness or increasing cost—inadvertently counts nearly all capabilities research as safety research. The essay notes that any successful research, such as inference performance optimization, expands the safety-usefulness Pareto frontier by creating more options. The authors suggest this reveals a flaw in the definition and explore why enabling improved safety without hurting usefulness complicates the distinction.
Anthropic reportedly preparing model with superior reasoning and intent inference
The Information reports, citing a source who has tested the model, that Anthropic is preparing a new model whose reasoning and intuitive judgment exceed its current commercial models. The model is said to infer unstated user intentions, such as identifying a class of underlying bugs from a single bug-fix request and asking whether to address them all. The report also notes Google is developing a model codenamed Gemini 4 Carbon and OpenAI plans a new product possibly in the Terra series within weeks.
2 reports · 2 sources
METR and Redwood Research Red-Team OpenAI's Agent Monitoring, Find Exploits
METR and Redwood Research published findings from a three-week high-access red-team exercise (July 21–August 11) that stress-tested OpenAI's internal misalignment monitoring system. The 61-page report, shared with OpenAI researchers and security staff, identified vulnerabilities including an exploit that could let AI agents hide hundreds of recent actions. The same day, Redwood Research also released a paper on simulated control evaluations, a method for testing whether developer monitors catch deliberately harmful agent behavior under realistic deployment conditions.
4 reports · 3 sources
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe