American company Anthropic has published a report on the misuse of its models, and one section of the report is particularly chilling. It describes five cases in which scientists and researchers used the Claude model while working on projects that could help develop biological weapons. The company blocked their accounts, shut down the networks they used to circumvent restrictions, and shared its findings with authorities and competing laboratories. What matters is how cautiously the company phrases this. It does not claim that any of these people intended to make a weapon. It says that it cannot determine this, and that is precisely why it intervened.
Not Just Bird Flu Experiments
The first documented case occurred in May this year. The security system Anthropic uses to monitor sensitive biological queries detected a request for help writing a grant application. It concerned research into the mosquito-borne chikungunya virus, for which there is no approved treatment. Those infected can experience symptoms for weeks or even months.
The proposed work was intended to modify the virus so that it could spread more effectively while also evading the immune system more successfully. The company rejected the request and, upon further investigation, discovered something else. The researchers in question were affiliated with a military research institute, which attracted additional scrutiny. But the matter did not end with a single rejection. Anthropic later discovered that members of this group were using a third-party service that circumvented geographic restrictions and automatically forwarded rejected queries to another model from a different company.
The second case unfolded more slowly and was all the more disturbing for it. One researcher spent weeks using the Claude model to plan experiments involving bird flu. He was interested in how to adapt the virus to mammals and how to make it spread through the air. He lived in a region where Anthropic restricts access to its models, so he accessed them through a rented server in another country. The monitoring systems eventually detected him, and the company cut off his access to its most powerful models. He was left with only the weakest ones, which have much more limited capabilities, but ultimately he was banned entirely.
The remaining three cases involved a virus related to smallpox, poisons, and toxins. In one of them, a researcher had the Claude model compile an extensive overview of toxic substances from various groups of venomous animals. Another involved redesigning toxins for a particular government program.
True Intent Is Difficult to Assess
Throughout this matter, Anthropic has encountered a problem that the field of biological safety has grappled with for decades. The same knowledge can be used for good or for harm. Information that can help build a weapon may just as easily serve to develop a vaccine or medicine, and it is often impossible to distinguish between these two intentions.
Moreover, none of these people wrote that they were planning anything malicious. On the contrary, the queries looked like ordinary scientific work: grant applications, experiment planning, and data analysis. Jacob Klein, who leads Anthropic’s threat assessment team, said that none of them had declared malicious intent and that the company did not know whether the intended outcome was a weapon.
In the report itself, the company adds that experienced attackers are aware of this ambiguity and deliberately exploit it to maintain plausible deniability. Ultimately, Anthropic chose to err on the side of caution, arguing that the consequences of overlooking such activity would be too serious.
What This Means for the Future Use of AI
The report contains one observation that goes beyond these cases. The company writes that its older models were clearly below the threshold at which they could genuinely help someone develop a biological weapon. With newer models, it is no longer certain. For the first time, a major company in the field is publicly acknowledging that it can no longer rely on its technology simply not being capable enough to be dangerous. Anthropic responded by introducing stricter safeguards for its latest models and restricting access to a broad category of biology-related queries that could be used for either beneficial or harmful purposes.
This point shows how the story affects everyone who uses artificial intelligence. The report highlights three issues that the industry will have to address. Blocking access at a single company is not enough. A rejected query can be sent elsewhere, and services exist that do this automatically. Until laboratories share information about whom they have blocked and why, circumventing restrictions will remain a matter of minutes. Geographic restrictions are not particularly effective either. A server in another country costs only a few dollars a month and can bypass national boundaries in moments.
Above all, however, the nature of safeguards is changing. As long as models were unable to assist with sensitive work, it was enough to monitor explicit calls for violence. Now that models can perform genuine scientific work, companies must also assess who is asking and why, which is incomparably more difficult. Many people conducting legitimate research will encounter these stricter safeguards and will be justifiably frustrated, because it is difficult to distinguish between a genuine scientist and someone merely posing as one. Access to models this capable should therefore be limited to vetted researchers.
Growing Concerns About Advanced Models
The biological section is only one of seven chapters. In the same document, Anthropic describes a Russian group building a swarm of attack drones, Chinese and Yemeni cases involving work on weapons software, and a user linked to Iran who collected publicly available data and used it to prepare recommendations for attacks on U.S. naval forces in the region.
Two days before the report was published, researcher Jacob Coxon also left Anthropic after publicly accusing both his former and previous companies of racing toward superintelligence and gambling with human lives.
U.S. President Donald Trump dismissed the growing concerns surrounding artificial intelligence on Thursday evening.
Sources: nytimes.com, cnn.com, bbc.com and foxbusiness.com
