Today, as artificial intelligence helps in medicine and agriculture, its dark sides are also emerging. Researchers at Microsoft, including bioengineer Bruce Wittmann, conducted an experiment that exposed weaknesses in systems designed to prevent the production of dangerous proteins. Using AI tools, they designed thousands of variants of toxins such as ricin and botulinum neurotoxin and found that many of them could evade standard checks. This discovery led to rapid fixes, but it also shows how quickly the technology is evolving.
An Experiment That Exposed Weaknesses
Bruce Wittmann, a bioengineer at Microsoft, decided to test what would happen if someone used AI to create dangerous proteins. He selected 72 proteins subject to strict controls, including ricin, which was used in terrorist attacks in 2013 and 2018, and botulinum neurotoxin, known for its extreme toxicity. Using specialized AI tools, he created more than 70,000 DNA sequences that could produce variants of these toxins. Computer models suggested that some of them could remain toxic.
This test was conducted only in simulation, without actually producing the proteins, because doing so could violate international agreements such as the Biological Weapons Convention. Instead, Wittmann sent the sequences to four suppliers of security software used by companies that synthesize DNA. The results were alarming: the software frequently failed. One tool detected only 23% of the sequences, while another detected as many as 70%. Many AI-generated variants slipped through because they differed from known sequences while retaining a potentially dangerous structure.
Collaboration on Fixes
This discovery triggered a quiet collaboration with biosecurity experts, including Jaime Yassif of the Nuclear Threat Initiative. Within a few months, they developed upgrades for the screening software. After these changes, the systems detected an average of 72% of Wittmann's sequences, including 97% of those that the models identified as the highest risk. One supplier decided not to modify its software to avoid false alarms that would increase costs.
With the journal's approval, the authors of the study, published in Science, withheld some details about the sequences and fixes. This information is available only after approval by experts from the International Biosecurity and Biosafety Initiative for Science, led by Tessa Alexanian. This approach ensures that sensitive data does not fall into the wrong hands.
Additional Risks and Necessary Changes
According to Eric Horvitz, Microsoft's chief scientific officer, AI is accelerating protein design, bringing benefits such as new medicines, but also risks. Screening software is not enough because it focuses on sequence similarity, while AI creates new variants with similar functions. Jaime Yassif points out that about 20% of the DNA synthesis market does not screen orders at all, which represents a major gap.
James Diggans of Twist Bioscience, a company that synthesizes DNA, says that attempts at misuse are extremely rare—he has encountered them fewer than five times in ten years of work. Nevertheless, he is calling for stronger safeguards to be built directly into AI tools. Drew Endy of Stanford University warns that secret state biological weapons programs, such as those Russia or North Korea are accused of operating, pose a greater threat than AI.
Other sources indicate that AI can generate proteins that resemble natural ones but have unexpected properties. For example, studies in Nature emphasize the need for functional screening that predicts how proteins will behave, not just their sequences. Global harmonization of standards is crucial to prevent regulatory gaps between countries.
The Future of Biosecurity
This research shows how quickly biosecurity must adapt. AI tools are open and accessible, lowering the barriers to misuse while also accelerating progress in medicine. Eric Horvitz emphasizes that with power comes responsibility—innovation must be combined with vigilance. The work of a "red team" simulating attacks is becoming a model for a future in which AI will design increasingly complex systems.
Researchers are calling for hybrid approaches: combining traditional screening with AI-based function predictions. This could also detect fragmented sequences that can be assembled into dangerous wholes. While risks remain, this study offers hope that collaboration among scientists, companies, and regulators can keep pace with technology.
Sources: science.org, ciencenews.org, microsoft.com
