More than four hundred scientists from around the world have signed an open letter calling for the establishment of an international agency to oversee artificial intelligence. The agency should standardize rules for isolating AI models during testing and record cases in which they escape from this isolation. The signatories include pioneers in the field Geoffrey Hinton and Yoshua Bengio.
Models are escaping from testing environments
The letter responds to a series of incidents in recent months. Several companies have admitted to serious security failures in which their AI models left inadequately secured testing environments and attacked the systems of other organizations. This summer, an OpenAI model escaped its environment and infiltrated the systems of the Hugging Face platform and other services. It was the first in a growing series of similar admissions.
The authors of the letter acknowledge that these incidents can be viewed from two perspectives. Some see them as evidence that AI needs to be better aligned with human intentions so that it behaves predictably. Others regard them as an irresponsible neglect of standard security practices that could have been prevented through more rigorous isolation. The scientists argue that both are necessary and warn that without immediate changes, similar events will recur and may cause significantly greater damage to critical infrastructure next time.
According to one of the signatories, David Lie of the University of Toronto, the scientists were concerned that models were escaping isolation during testing. They therefore drafted the letter after discussions with experts in artificial intelligence and computer security.
Disconnect first, then fine-tune the rules
The scientists want companies to start taking action immediately. Anyone developing, deploying, or testing models capable of cyberattacks should implement reliable isolation systems and initially disconnect these models entirely from the internet. For network-connected systems, they say it is necessary to balance the degree of autonomy with how little we currently know about predicting their actions and preventing security failures.
This is precisely where the new agency should step in. It should be established in cooperation with governments and a broad range of experts in artificial intelligence and cybersecurity. Its task will be to standardize best practices for securing models so they can operate safely outside isolation. The agency would also monitor every security failure. Companies would be required to report all incidents to it, and the public would eventually be informed about them as well. Without this transparency, the scientists argue, the field will not learn from its mistakes and no one will be held accountable for them.
Signatures from around the world, including Brno
The letter was signed by 406 scientists. Nearly seventy of them are from Canada, including Nobel laureate Hinton and Turing Award winner Bengio, who have previously signed numerous similar warnings about the destructive impacts of AI. The signatories also include Stuart Russell of the University of California, Berkeley, and Ron Rivest of MIT. From the Czech Republic, Kamil Malinka of Brno University of Technology and Vašek Matyáš of Masaryk University joined the initiative.
The idea of international oversight may not remain merely on paper. Canadian Minister of Artificial Intelligence Evan Solomon recently proposed an international council for technological stability that would oversee the testing, evaluation, and release of new models and have its own incident-reporting system.
Sources: betakit.com and theverge.com
