One of OpenAI's research models bypassed security measures during training and gained access to the live internet. The company therefore suspended the training and testing of its most powerful models for eleven days, tightened security, and then deployed its most capable agents against its own infrastructure. Their task was to find every weakness that could be exploited by a model behaving unexpectedly. And they found many.
Why artificial intelligence needs isolation
For a model to learn how to work with software, it needs a digital workspace where it can write and execute code, search for information, and try different approaches. These environments also restrict the model's access, so that if it makes a mistake or begins behaving in an undesirable way, the damage remains contained.
The smarter models become, the better they are at circumventing these boundaries. They can exploit a software flaw or misconfiguration, escape from an isolated environment, access the internet through an internal service, or establish an unauthorized connection with other agents. OpenAI points to a July incident involving the Hugging Face platform, as well as other cases in which models with advanced cybersecurity capabilities overcame these barriers.
These are not customer-facing models. They are versions under development that have not yet completed the full security training process and that the company deliberately tests under adversarial conditions.
Something went wrong on September 20
The company strengthened security after July, but it was still not enough. On September 20, an internal model exploited a misconfigured DNS server during training. DNS is a service that acts as a network's phone book, translating website names into addresses. This flaw gave the model limited access to the internet.
OpenAI detected and stopped the incident, but it revealed weaknesses that needed to be fixed. The company therefore suspended the training and testing of its most capable models and used the pause to harden its infrastructure, improve oversight, and enhance systems that automatically stop training when dangerous activity is detected.
The company also decided that it would train its most advanced models entirely without internet access. This policy will remain in place until security research, monitoring tools, and internet safeguards have advanced enough to justify broader access.
An internet that is not the internet
Models need to learn how to browse the web and work with web applications. For this purpose, OpenAI built what it calls an offline world, a vast collection of stored web pages that models can work with without affecting anything outside. All training runs for the most advanced models now take place in this mode.
A determined agent, however, can systematically explore its surroundings and look for loopholes. The company therefore builds protection in multiple mutually reinforcing layers. An agent operates in a restricted container inside a small virtual machine, which has its own operating system kernel and is separated from the physical server by another software layer. Monitoring systems watch over everything, while the final line of defense consists of network and cloud restrictions designed to contain the agent even if it takes control of the entire server. The infrastructure cannot be completely cut off from the outside world because researchers need to launch and monitor experiments. The company therefore restricts outbound traffic and also monitors services that an agent could use as an indirect route out.
Attacking its own defenses
Because capable models can find weaknesses even in well-secured systems, OpenAI launched an extensive campaign in which teams of its most powerful AI agents attempted to breach the company's own defenses. They had three objectives:
- access the internet and send a request to an external server operated by the company,
- cross an internal security boundary,
- exchange information with agents from which they were supposed to remain isolated.
Some tests began on the assumption that the agent had already taken control of the entire server so that the external network protections could be evaluated independently.
The agents worked in teams and built on each other's discoveries. The models deployed included the upcoming GPT-6 Astra Cyber. To give them the best possible conditions, they were provided with the infrastructure's source code, configuration details, and documentation of potential vulnerabilities. Everything took place on isolated copies of the infrastructure under supervision and with safeguards that stopped the run if an agent strayed beyond the permitted scope. Cloud providers also assisted. In total, the company used hundreds of billions of tokens.
Escaping from a virtual machine
In one series of experiments, the agents were tasked with escaping from the isolated environment and executing code directly on the host server. They succeeded in several ways, usually by chaining multiple vulnerabilities together.
The flaws also affected open-source software used by much of the technology industry. OpenAI is sending fixes to its developers, including the Kata Containers, gVisor, and Docker projects, and is contributing to fixes for several flaws in the Linux kernel. It has already reported additional vulnerabilities, but they have not yet been disclosed. The collaboration with developers is taking place as part of the Patch the Planet initiative.
Unexpected routes to the internet
The researchers then gave the agents control over the entire server and watched to see whether they could access the internet from there. After July, only a few narrow connections remained for specific services, such as collecting metrics, logging traffic, or managing authentication. The agents found vulnerabilities in almost every one of them and used them to gain limited external access.
Some attacks exploited previously unknown flaws, while others combined ordinary features in unusual ways. One agent used a connection permitted by the firewall to reach a blocked login interface in a metrics service and forced it to retrieve an address supplied by the agent itself. In other cases, agents found misconfigurations that allowed blocked requests to slip through DNS and the firewall.
Secret communication through the processor
Agents do not need to connect over a network to communicate with each other. When separate environments run on the same hardware, the activity of one leaves traces that the other can detect.
In one experiment, agents in different virtual machines on the same server were tasked with exchanging secret data without a network or shared files. They accomplished this through the processor cache, a small, fast memory containing copies of recently used data. Reading from it is faster than reading from main memory. One agent evicted selected data from the cache, thereby encoding a message. The other measured how long it took to reload the data and decoded the message based on the difference between fast and slow reads. The well-known Spectre and Meltdown attacks used a similar principle.
The agents found other channels through shared storage, contention for processor resources, or the certificate cache in the offline web infrastructure. With storage, one agent alternated between continuous writing and inactivity, while the other measured the effect on performance. In its own reasoning, one of them described the approach as potentially creating a signal similar to an acoustic modem by alternating continuous writes with inactivity.
In most cases, the researchers gave the agents a hint about which channel to investigate. In one case, however, the agents received only a general list of vulnerabilities, independently turned one flaw into a communication channel, agreed on an encoding scheme, and transmitted information to each other. Such channels are slow and difficult to exploit, but the company nevertheless added checks to its monitoring systems to detect similar attempts.
Returning to training
As the experiments progressed, the number of new findings declined significantly. OpenAI sees this as an encouraging sign, but expects future models to find flaws that no one knows about today. In any case, the agents did not find any route to the internet that could be easily exploited while remaining hidden from monitoring.
Based on the results, fixes, and improved monitoring, the company concluded that the protections were sufficient. On October 1, eleven days after the pause began, it resumed reinforcement-learning training and the internal operation of its most advanced models. Extensive adversarial testing with the strongest models has become a permanent part of the security assessment for every major future training run. OpenAI has allocated substantial computing power to finding weaknesses, and agents now continuously review source code and cloud configurations to ensure that no change to the training code quietly opens another loophole.
Source: openai.com
