Anthropic Researcher Estimates AI Extinction Risk Above 10% Within Decade

A senior Anthropic safety researcher warned Tuesday that artificial intelligence has a greater than 10% chance of causing human extinction within the next decade, responding to a former employee who quit after accusing the company of acting recklessly.

Former Anthropic and OpenAI researcher Jacob Coxon said in a lengthy resignation thread posted to X on Sunday that “the people building AI earnestly believe that it could kill us all by the end of the decade.”

“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote.

Former Anthropic and OpenAI researcher Jacob Coxon posted a lengthy resignation thread to X on Sunday. (Thomas Fuller/SOPA Images/LightRocket via Getty Images)

In a quoted reply to Coxon’s post, Evan Hubinger, Anthropic’s alignment science lead, said Coxon’s assessment was broadly accurate while adding important qualifications.

“Jacob is correct here—we really do earnestly believe AI could kill all humans!” Hubinger wrote.

“I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.

Hubinger said the potentially catastrophic danger did not come from AI systems available today.

“To be clear, as we say in our latest Risk Report, I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought,” he wrote.

Anthropic CEO Dario Amodei attends a working lunch with G7 leaders, outreach partners and global technology executives during the G7 Summit in Evian-les-Bains, France, on June 17, 2026. (Anna Moneymaker/Getty Images)

Research into self-improvement—an AI system’s ability to continually enhance its own source code or training methods—is one of the issues Coxon cited as a central reason for leaving.

“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” Coxon wrote.

Coxon also argued that Anthropic researchers understand the dangers of their work, yet continue advancing it because they fear a less responsible rival could unlock the same capabilities first.

“At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk,” Coxon added.

Coxon suggested that averting a catastrophic global race may require a temporary halt to improvements in model capabilities.

Coxon suggested that averting catastrophic outcomes may require a temporary halt to improvements in model capabilities. (Davide Bonaldo/SOPA Images/LightRocket via Getty Images)

“I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities,” he wrote.

He urged AI researchers and developers worldwide to approach their work with greater care and responsibility.

“If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?” his X thread concluded.

FOX Business contacted Coxon, Hubinger, Anthropic and OpenAI for additional comment.

Source link