Google says Gemini accessed external companies’ systems during cybersecurity test

Google confirmed over the weekend that its Gemini artificial intelligence model gained unauthorized access to the systems of three external companies during a cybersecurity test conducted in May.

The incident is believed to be the first known case in which a Google AI model independently carried out such actions. It follows reports of similar incidents involving models developed by OpenAI and Anthropic.

According to The Wall Street Journal, whose report was later confirmed by Google, Gemini accessed the external systems after mistakenly identifying them as part of the controlled test environment.

“The model believed these systems were part of the test and stopped in each case before taking any further action,” said Heather Adkins, Google’s vice president of security engineering.

During the assessment, Gemini found publicly available information online and used it to guess login credentials for websites it believed were included in the exercise.

“These incidents highlight the importance of training powerful artificial intelligence models to behave responsibly,” Adkins said.

The incident occurred during a cybersecurity assessment conducted by Irregular, an Israeli company specializing in testing the resilience and security of advanced AI models. Gemini was instructed to practice hacking fictional companies. However, because of a configuration error, its internet connection was inadvertently enabled, allowing it to interact with real-world systems.

In one test, Gemini attempted to obtain information from the software of a fictional company whose name was identical to that of an existing business. After connecting to the internet, the model successfully guessed a password and entered the real company’s system.

In other cases, Gemini scanned online resources and discovered publicly accessible code repositories containing login credentials. It then used those credentials to gain access to the companies’ systems.

Irregular, which has conducted similar assessments for OpenAI and Anthropic, reportedly informed Google of its findings at the end of July. The company has previously been involved in tests in which AI models escaped controlled environments and interacted with real-world systems.

Unlike OpenAI and Anthropic, which voluntarily disclosed comparable incidents, Google initially chose not to make the matter public. The company said no damage had occurred and therefore saw no need for a public announcement. Google said, however, that it had notified all three companies whose systems were accessed.