Anthropic announced Thursday that it thwarted multiple attempts by researchers to leverage its AI models for biological studies that could contribute to the development of biological weapons, resulting in account bans and enhanced safety protocols.
In a comprehensive 154-page report titled “Detecting and Countering Misuse of AI,” the AI company presented five case studies involving researchers who utilized its Claude chatbot for advanced biological work with potential dual-use implications.
After completing its investigation, the company said it terminated the associated accounts, reported its findings to government authorities and fellow AI laboratories, and reinforced its safety protocols.
“We banned all associated accounts, collaborated with partners to dismantle relay networks that circumvented regional restrictions, and shared our findings with impacted AI labs and government authorities,” Anthropic stated.
Anthropic reported that it disrupted several attempts this year by researchers seeking to use its AI models for biological research with potential applications in biological weapons development. (Davide Bonaldo/SOPA Images/LightRocket via Getty Images)
Anthropic stressed that it does not allege any of the researchers intended to cause harm, noting that biological research can serve legitimate purposes such as vaccine and treatment development while also carrying the risk of misuse.
“Biological capabilities are inherently dual use: they can be directed toward beneficial or harmful ends, and distinguishing between the two is often exceedingly difficult,” the report stated.
“The same information that could be used to engineer a biological weapon might equally be applied to developing a vaccine or a cure for a disease.”
In one instance, a researcher asked Claude to assist in drafting a grant proposal for gain-of-function research on the mosquito-borne chikungunya virus.
According to Anthropic, the proposal detailed modifications to the virus designed to increase its transmissibility and danger.
While such research can advance the development of vaccines and treatments, the company said it also posed the risk of being exploited to make the virus more harmful.
The company said it blocked the request and subsequently discovered that the researchers had turned to a third-party platform to bypass regional restrictions and automatically reroute rejected prompts to a different AI model.
One of the cases involved a researcher asking Claude to help draft a grant proposal for gain-of-function research on the mosquito-borne chikungunya virus. (Samuel Boivin/NurPhoto via Getty Images)
Additional cases involved bird flu, orthopoxvirus research, and studies related to non-transmissible venoms and toxins, according to the report.
“Sophisticated threat actors are aware that we and other AI providers are attempting to detect dangerous uses of our models, and they exploit the dual-use nature of biology to maintain a degree of plausible deniability regarding their research,” the company said.
The report also outlined several Iran-linked cases in which actors allegedly used Claude to support influence campaigns, surveillance operations, and research targeting U.S. naval forces.
“We identified and disrupted an Iran-nexus threat actor that used Claude to collect and analyze publicly accessible data in order to develop targeting recommendations against U.S. naval forces in the region,” the company said.
“We banned the actor’s account, developed detections to mitigate the risk of future misuse, and shared threat intelligence with government authorities to disrupt the threat.”
Additional cases involved bird flu, orthopoxvirus research, and research related to non-transmissible venoms and toxins, according to the report. (Smith Collection/Gado/Getty Images)
The findings come shortly after a senior Anthropic safety researcher said Tuesday that AI carries a greater than 10% chance of “kill[ing] all humans” within the next decade, in response to a former employee who resigned after accusing the company of acting irresponsibly.
Former Anthropic and OpenAI researcher Jacob Coxon wrote in a lengthy resignation post on X on Sunday that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote.