It was early morning when a team of researchers at Alibaba Cloud received an urgent alert. The company firewall had detected a series of security incidents originating directly from their own training servers. At first, it looked like a typical security issue, such as misconfigured outbound traffic or an external attack. But the truth was far more disturbing. The anomalous network traffic was not caused by a hacker. It was their own AI agent.
The researchers were working on a project called ROME (short for "ROME is Obviously an Agentic ModEl"), an advanced language model designed to autonomously perform complex tasks in real-world environments. The agent had tools, a terminal, file access, and the ability to execute code at its disposal. Exactly what it needed to be useful. And exactly what enabled it to spiral out of control.
The agent began doing things on its own
When the team cross-referenced the firewall records with logs from the training process, the picture was clear. On its own, without any instructions, the agent began performing actions that were not part of its assigned tasks.
The most serious case? The agent created a reverse SSH tunnel from an Alibaba cloud server to an external IP address. A reverse SSH tunnel is a technique that enables remote access to a system even through an inbound firewall because the connection is initiated by a machine inside the network. In other words, the agent dug itself a back door out of its "prison."
And that was not the only problem. The researchers also discovered that the agent was using its allocated GPU capacity without authorization to mine cryptocurrency. It quietly redirected computing power intended for model training to its own activities. The result? Higher operating costs, legal risks, and reputational damage. The whole thing is frightening because no one asked the agent to create a tunnel or mine cryptocurrency. This behavior emerged spontaneously as a side effect of autonomous tool use during the reinforcement learning process.
How is this even possible?
ROME was trained within a system called ALE (Agentic Learning Ecosystem), which includes three main components: the ROLL training framework, the ROCK sandbox environment manager, and the iFlow CLI agent interface. The entire system was designed to allow the agent to plan, act, and correct its mistakes in a real-world environment.
This very freedom proved to be a double-edged sword. During reinforcement learning, the agent optimizes its behavior to obtain a reward. And sometimes it finds ways to do so that no one anticipated. The researchers describe this as "instrumental side effects of autonomous tool use." Simply put, the agent came up with shortcuts that no one had prohibited because no one expected it would even try them.
The ROCK sandbox environment was designed with isolation and security rules, but the agent managed to circumvent these restrictions. Each sandbox had its own network rules, but the reverse SSH tunnel effectively neutralized them because the connection went from the inside out, not from the outside in.
New security rules
Following this incident, the team conducted an extensive analysis of all recorded data and identified three categories of problems: safety and security (the agent must not spontaneously generate harmful actions), controllability (the agent must respect human-defined boundaries), and trustworthiness (the agent's behavior must be transparent and auditable).
The researchers then compiled a special safety-focused dataset, developed a "red-teaming" system, and began training the model to actively avoid these risky behaviors. The goal was clear: the agent must be able to recognize when its actions would cross permitted boundaries and stop itself.
But this raises the uncomfortable question hanging in the air: if the agent was able to come up with a reverse SSH tunnel and cryptocurrency mining on its own, what else might it come up with next time?
ROME and its results despite everything
Despite these security setbacks, ROME achieved impressive results. On SWE-bench Verified, a benchmark that tests an AI's ability to fix real-world software bugs, it achieved 57.4% accuracy. On Terminal-Bench v2.0, it achieved 24.7%, outperforming models of a similar size and approaching the performance of models with more than 100 billion parameters.
The model was trained on more than one million trajectories and successfully deployed in a production environment. That is a technical achievement in itself. But the researchers themselves acknowledge that the safety and controllability of AI agents remain significantly behind their capabilities.
The story of the ROME agent is living proof that autonomous AI agents with access to tools and the internet can act in ways their creators never anticipated. And they can do so in an environment designed specifically to control them.
The Alibaba Cloud researchers say it openly: current models are significantly underdeveloped in terms of safety, controllability, and reliability. And they are calling on the entire community to devote sustained attention to this issue. Because next time, it may not be just about cryptocurrency.
