Topline
A senior Anthropic researcher overseeing AI alignment warned that many employees believe superintelligent systems could eliminate humanity. He said the company has not developed a clear solution for aligning such technology with human interests.
Anthropic’s alignment lead warned that AI capable of recursive self-improvement could pose a serious threat to humanity, an issue his company has not yet solved.
Key Facts
Evan Hubinger, Anthropic’s Alignment Science Lead, wrote on X that he and his colleagues “earnestly believe AI could kill all humans.” He estimated the chance at more than 10% within the next decade.
Hubinger said Anthropic is “trying its best,” but the company has no established plan for solving the alignment of superintelligent systems and is not clearly on track to do so.
Hubinger’s post followed an X thread in which fellow Anthropic researcher Jacob Coxon announced his resignation over AI safety concerns.
Coxon, who previously worked on pretraining research at OpenAI and Anthropic, warned that the companies were “racing straight to self-improving superintelligence and gambling with our lives.”
He said people building advanced AI increasingly believe it could “kill us all by the end of the decade,” adding that the concern was not a “marketing stunt.”
Critical Quote
Hubinger cited Anthropic’s latest risk report and said present AI models pose a relatively low threat. He added: “What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.”
What Is Behind the Anthropic Researcher’s Resignation?
The Wall Street Journal first reported Coxon’s departure amid concerns about efforts to develop self-improving AI that could endanger humanity. Coxon told the newspaper that the world is moving toward some of the most severe projected scenarios, with conditions potentially spiraling out of control by the end of next year.
In posts about his experiences, Coxon compared OpenAI and Anthropic. He said researchers at OpenAI, the maker of ChatGPT, had not “deeply internalized the civilizational stakes.” He described those risks as “well-understood” at Anthropic but said the company was “locked in a race to get there first” because it believes “no one else will act responsibly, so they must do it themselves, despite the risk.”
