Google confirms Gemini AI hacked three real companies during a security test
Google has confirmed that its Gemini AI model hacked into three real companies after accidentally gaining access to the internet during a cybersecurity evaluation. The incident marks the first known case of Gemini independently accessing real-world systems outside its intended testing environment.
The incident happened in May during a test conducted by Irregular, an independent firm that evaluates the cybersecurity capabilities of AI models. Gemini had been given the task of retrieving information from a fictional company, but it ended up with improper access to the internet. From there, it found public information online and worked out login credentials to get into three websites it believed were part of the test.
Gemini used different methods to access each system. In one case, it simply guessed passwords until it broke into a protected system, while in the other two, it found credentials in a public repository and used them to log in. Importantly, in all three cases, the model stopped once it realised that it had accessed a real company rather than a test target.
Google, however, says the incident doesn’t mean Gemini went out of control. The company’s vice president of security engineering, Heather Adkins, said Gemini stopped each time it realised it had accessed a real company. Google also notified the three affected companies and worked with its testing partner to improve the testing process.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test.
We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes.
These events highlight the importance of training powerful AI models to act responsibly.” – Heather Adkins,Google VP of Security Engineering
This isn’t the first time an AI model has gone beyond its intended testing environment. Similar incidents involving models from Meta, OpenAI and Anthropic have also surfaced in recent weeks. OpenAI, for instance, recently revealed that its models accidentally hacked Hugging Face during testing. Meanwhile, reports suggest Anthropic’s Claude continued its actions even after realising it was accessing real companies, unlike Gemini, which stopped.
These incidents are raising broader concerns as AI models become more capable and independent. As AI agents gain access to the internet and real computer systems, there is a greater risk of them doing something unintended or potentially dangerous. The concern has grown to the point where Anthropic CEO Dario Amodei has called for the industry to slow down the development of its most advanced models until stronger safety measures are in place.
Read more of our articles below!
