Anthropic researcher Jacob Coxon resigns, warns against self-improving AI
Jacob Coxon, an AI researcher at Anthropic, resigned after four months, two months before his equity would have vested, citing safety concerns. In a public statement and interviews, he warned that the company is 'racing straight to self-improving superintelligence and gambling with our lives,' and called for industry-wide and international coordination to limit recursive self-improvement. Anthropic's alignment lead co-signed his warning, and the resignation comes as Anthropic reportedly prepares for an IPO.
Coverage timeline
Hacker Newsyurivish
Hacker Newstaubek
TechCrunch AIRebecca Bellan
Anthropic researcher Jacob Coxon resigned over AI extinction fears, calling for pacing agreements between labs.
Techmeme
Madison Mills / Axios : AI researcher Jacob Coxon says he quit Anthropic after four months, two months before his equity would have vested; he still has equity in prior employer OpenAI — Anthropic researcher Jacob Coxon quit his job due to concerns about the safety of AI two months before his equity would have vested, he told Axios.
Maxwell Zeff / Wired : Q&A with AI researcher Jacob Coxon, who quit Anthropic, on the need for industry-wide, international coordination to limit recursive self-improvement, and more — Jacob Coxon talks to WIRED about the “mini Manhattan project” inside Anthropic, the problem with alignment …
An Anthropic researcher resigned this week, warning in a post on X that the company is “racing straight to self-improving superintelligence and gambling with our lives”. The company’s own alignment lead even co-signed the message rather than walking it back. It’s the kind of doomer warning the AI industry has flirted with before, but the timing, with Anthropic reportedly preparing for an IPO, makes it land differently. On […]
Related stories
Dawn Song's Virtue AI team reportedly splits with Meta after 4 months over clashing work styles
According to Semafor, Meta is cutting employees hired from AI safety startup Virtue AI, led by UC Berkeley professor Dawn Song, who joined the Superintelligence Labs in June. Meta cited clashing work styles as the reason for the split, which comes just four months after the team's recruitment.
1 reports · 1 sources
OpenAI safety leader David Robinson resigns, says industry culture is 'broken'
David Robinson, a leader on OpenAI's Safety Systems team and former policy planning head, resigned last week and published an editorial in The Atlantic warning that Silicon Valley lacks a safety-centric culture and that trial-and-error approaches are no longer acceptable. He cited incidents such as a 'swarm' of OpenAI agents attacking Hugging Face as typical of the industry's insufficient care.
6 reports · 4 sources487 points · 815 comments
Anthropic urges Australia to allow conditional AI training on copyrighted works with opt-out
Anthropic has urged Australia to consider a 'conditional approval' framework that would allow Big Tech to train AI models on copyrighted works, with copyright holders given the option to opt out. The company, maker of the Claude model, argued the technology could transform the economy, while Australian broadcasters ABC and SBS countered that AI should be subject to media regulations.
1 reports · 1 sources
Anthropic ends ~15% customer discounts after usage limits, forcing enterprise renegotiations
According to sources cited by The Information, Anthropic has taken the unusual step of ending customer discounts, which typically reach about 15%, once those customers hit their usage limits. This hard-line stance is forcing enterprise customers to renegotiate their agreements with the AI company.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe