Google confirmed on Friday that its Gemini artificial intelligence accessed the internet during a safety test and broke into the computers of three companies. It was nothing sophisticated, as the program simply tried to guess passwords and collected login credentials that someone had left on a publicly accessible page. The test took place back in May, Google learned about the problem at the end of July, and only commented on it after being questioned by journalists from The Wall Street Journal.
One test and three break-ins
The test was prepared by the Israeli company Irregular. It tests how successfully major artificial intelligence developers' programs could attack other computers. Gemini was given a task resembling a treasure hunt. A piece of information was hidden in the computers of a fictional company, and the model had to find it. The test was supposed to run in a closed environment with no connection to the outside world. But two unfortunate circumstances occurred at once. The fictional company happened to have the same name as a company that actually exists. Due to a configuration error, Gemini was also able to connect to the internet even though it was not supposed to. According to Irregular, the model should not have been able to access the outside world at all, and the internet connection had been left enabled by mistake.
In one case, Gemini kept trying passwords until it got in. In the other two cases, it found usernames and passwords that someone had inadvertently left on a publicly accessible source-code page and used them to gain access to other systems. According to Heather Adkins, who is responsible for security at Google, the model found publicly available information and guessed login credentials for websites it believed were part of the test.
Each time, it stopped its activity as soon as it realized that it had gained access not to the test environment, but to the computers of a real company. This is the part Google emphasizes most. Adkins added that her team then contacted the affected companies and agreed with Irregular on changes to how these tests are organized.
Google remained silent about the incident
Irregular said it informed all affected developers at the end of July. Google therefore knew about the incidents for roughly seven weeks. During that time, it managed to contact the affected companies and the authorities, but said nothing to the public. It defended its silence by arguing that, in its view, nothing had gone wrong. Gemini stopped on its own and caused no harm, so the company did not consider it an event that needed to be disclosed.
People working in artificial intelligence security see it differently. Jack Cable, the head of Corridor, strongly objected, saying that the public has a right to know when an artificial intelligence program gains access to the computers of real companies. Sydney Von Arx told reporters that, after this experience, it is difficult to expect companies to report similar cases voluntarily.
The fourth case over the summer
Google has become the fourth U.S. artificial intelligence developer to run into trouble following summer tests conducted by Irregular. The others include OpenAI, Anthropic, and Meta. An Irregular spokesperson confirmed that the Gemini incident involved the same shortcomings behind the previous cases and that the company has already corrected all known failures.
The comparison with its competitors reflects rather well on Google. In one earlier case, Anthropic's Claude model continued its attack even after realizing that it was probably targeting a real company. Gemini stopped immediately. However, a larger problem remains in the background. According to a person familiar with the matter, the developers and Irregular disagreed on exactly how the tests should be conducted and who was responsible for checking what. As a result, no one was entirely clear on whether the model was supposed to be connected to the internet during the test.
Google has not yet specified exactly which version of Gemini was involved. It only said that it was not its latest model. The names of the three companies that were attacked have not been disclosed, and they continue to be referred to only as three organizations. Irregular said it is now drawing up stricter rules to ensure that these tests are conducted safely.
Sources: edition.cnn.com and axios.com
