Sam Altman and Dario Amodei do not particularly like each other. At a summit in New Delhi, they stood in a row of fourteen people for a group photo and were the only ones who did not shake hands. Altman publicly admitted that the announcement of OpenAI's collaboration with the U.S. Pentagon looked "opportunistic and sloppy." Amodei, meanwhile, declared that he could not "morally capitulate" to the Defense Department's demands. And yet both signed a joint open letter to the U.S. Congress.
They were joined by Demis Hassabis of DeepMind and Mustafa Suleyman of Microsoft. Along with more than fifty other figures from the fields of artificial intelligence, biotechnology, and national security. The message states: artificial intelligence is becoming a threat that could erase the barriers preventing terrorists from producing biological weapons. And Congress must act before it is too late.
What exactly the letter says
The letter, organized by the Institute for Progress and the conservative-leaning Foundation for American Innovation, is not asking for anything vague. It calls on the U.S. Congress to pass a law requiring companies that sell synthetic DNA and RNA to screen both customers and orders. And, in addition, to keep records of exactly what they sold and to whom. "Awareness of traceability alone deters misuse," the letter states.
The signatories warn that artificial intelligence systems now outperform virologists with PhDs when answering highly technical questions in their own field. The knowledge barriers that have historically prevented malicious actors from gaining access to biological weapons are beginning to crumble. And fewer and fewer of them remain.
Synthetic DNA: cheaper, faster, and more accessible
To understand the risk being discussed, it is necessary to know how synthetic biology works. American biochemist Arthur Kornberg was the first to successfully synthesize DNA in the 1950s. Today, the entire process is automated. Dozens of companies around the world use commercial synthesizers to "print" and sell custom genetic sequences. They are used by scientists, biotechnology companies, and educational institutions. But not all companies screen their customers or the sequences they order.
The year 2017 showed just how real this threat is. Canadian researchers then reconstituted the extinct horsepox virus for $100,000 using DNA ordered by mail. Critics warned at the time that the same method could also be used to construct smallpox. Since then, the cost of gene synthesis has only continued to fall.
The combination of inexpensive DNA synthesis and powerful artificial intelligence now makes it practically possible to design dangerous toxins and pathogens even without an in-depth education in biology. David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter, says of this: "AI tools can very quickly show users where to order sequences that are not subject to screening. And if you ask them the right way, they will advise you on how to modify an order so that even companies that do screen cannot detect it."
Voluntary measures are not enough
Part of the industry has been addressing the issue for years. The International Gene Synthesis Consortium was established in 2009 to introduce voluntary screening procedures. Many companies now use software that scans orders for "sequences of concern" capable of contributing to an organism's toxicity or its ability to cause disease. But voluntary measures have their limits. And artificial intelligence is exceeding them.
Last year, Microsoft researchers published a study showing that AI-based protein design tools can generate potentially dangerous gene sequences that evade screening software. The models design new sequences with structures similar to those flagged as dangerous, yet still different. Screening software simply fails to detect them. If you have technology capable of synthesizing DNA, you must ensure that it is used responsibly. That means knowing what you are producing and for whom.
And what about AI companies?
However, the letter deliberately avoids one sensitive question: whether the new rules should also apply directly to companies developing artificial intelligence. Some of them claim that they are already taking measures themselves to prevent their products from being misused. But Geoff Ralston, former president of Y Combinator, says that is not enough. "It must be very difficult, if not impossible, to ask a model for help with something immediately dangerous." Relman agrees: "Screening can fail. We therefore need other checkpoints as well. And that is where AI companies will have to step in."
According to Severance, companies could program chatbots to refuse queries about laboratory procedures that would enable even non-experts to manipulate dangerous biological agents more effectively. They could also restrict access to specialized tools for designing proteins and chemicals. "It is about finding a balance," she says. "Ensuring that good science can continue, while attempts to misuse biology are detected and stopped."
Biological weapons are rare in terrorist attacks, historically accounting for only 0.02% of all attacks. But they are unforgettable. In 2001, five people died and another 22 were infected after a series of anthrax-laced letters were sent to U.S. senators and media offices. The attack prompted one of the largest FBI investigations in history. Untreated inhalational anthrax is fatal in nearly 100% of cases.
Sources: wired.com, fortune.com, and science.org
