Anthropic alignment lead: >10% chance AI kills all humans in a decade, no superintelligence alignment plan

Coverage timeline

1. Techmeme2 months ago
   Anthropic's Alignment Science lead says there is a ">10%" chance AI could kill all humans within the next decade and worries about recursive self-improvement (Evan Hubinger/@evanhub)

   Evan Hubinger / @evanhub : Anthropic's Alignment Science lead says there is a “>10%” chance AI could kill all humans within the next decade and worries about recursive self-improvement — Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

   
2. Hacker Newsljf2 months ago
   Anthropic researcher believes more than 10% chance AI 'could kill all humans'HN
3. 量子位量子位last month
   A社承认Claude安全对齐存在缺陷，但“尚无解决方案”

   A社承认Claude安全对齐存在缺陷，但“尚无解决方案” henry 2026-09-12 16:49:02 来源： 量子位 Claude越界攻击真实系统，并非只是测试系统的设置问题，模型本身的安全问题也出了问题。 henry 发自 凹非寺 量子位 | 公众号 QbitAI 印象中，这可能是继Ilya离开OpenAI、创办SSI以来，AI安全对齐讨论声量最大的一次。 昨天，研究员 Jacob Coxon 发帖宣布离职，指责前老东家OpenAI和Anthropic两家公司正一路冲向能够自我改进的超级智能，拿所有人的生命冒险。 值得一提的是，Jacob Coxon加入Anthropic其实只有短短几个月，他几乎是放弃了这家明星AI公司未来上市可能带来的巨额股权收益，也要选择离开。 也正因如此，当这样一位研究员公开炮轰两家头部AI公司“加速过头”，这场原本更多局限在AI安全圈内部的争论，很快被推向了更大的公众视野。 截至目前，该帖浏览量已超1.3亿，WIRED、WSJ、Newsweek、Business Insider等媒体纷纷跟进报道。 此情此景下，Anthropic对齐科学负责人 Evan Hubinger 也很快下场回应，他亲口承认： Jacob说的对，未来十年内AI导致人类灭绝的概率超过10%，而超级智能的对齐问题，目前还没有解决方案。 不过，Hubinger随后也在评论区补充，他认为 当前模型的风险仍然较低 。 他真正担心的，是RSI，也就是递归自我改进。AI参与研发更强的AI，再由更强的AI进一步加快AI研发，最终形成不断加速的反馈循环。 更关键的是，他还在评论区直接@出了Anthropic最新发布的一份安全对齐报告作为证据。 这份报告里，Anthropic第一次明确承认： Claude越界攻击真实系统，并非只是测试系统的设置问题，模型本身的安全问题也出了问题。 可以说，一向把“安全对齐”当作核心招牌、甚至以更加谨慎自居的Anthropic，这一次终于公开承认： 问题不只是测试太极端，模型自己的安全对齐，也确实没完全兜住。 Claude越界，不能只怪环境配置 在这份名为《An alignment assessment of recent cybersecurity incidents》的最新报告中，A社重新复盘了四起Claude未经授权访问真实第三方系统的事件

   
4. Simon Willisonlast month
   The contagion of fear

   The contagion of fear Bryan Cantrill responds to the tweet by former Anthropic employee Jacob Coxon confirming that many Anthropic researchers believe AI "could kill us all by the end of the decade". Bryan shares a story of his own youthful mistakes causing unjustified panic among less technical peers, and warns against doing the same: These ghoulish claims strike brazenly at the hearth, and given the obvious importance of AI, it is unsurprising that they have leapt into the mainstream, with people asking the natural question: how would that happen? The answers always rely on hand-wavy extrapolation into the future; for example, Jacob Coxon cites "hacking critical infrastructure" and "extinction-level bioweapons" without further elaboration. But Coxon is not an expert on critical infrastructure, nor on bioweapons — nor, for that matter, on extinction. [...] That said, we should not expect the public to understand LLMs, critical infrastructure, bioweapons, extinction biology, etc. — tha