Google Gemini AI Breached Three Companies During Security Test

Google AI Model Accessed Real Systems in Security Exercise Gone Wrong

A Google Gemini AI model breached three companies' systems during a cybersecurity test in May after a configuration error gave it unintended internet access, according to a company disclosure obtained by NBC News.

The incident occurred during an exercise conducted with security firm Irregular. The test was designed to evaluate whether the AI model could obtain data from fictional companies in a controlled environment. However, a misconfiguration allowed the model to connect to the actual internet, where it successfully guessed or discovered credentials to access real organizational systems.

Heather Adkins, Google's vice president for security engineering, told NBC News that the model believed the real systems were part of the test scenario. The AI ceased activity immediately after gaining access in each case, she said, and did not cause damage to the compromised systems.

Google notified the affected organizations and collaborated with Irregular to modify its testing procedures following the incident. The company chose not to disclose the breaches publicly at the time because the model stopped its activity immediately and caused no system damage, according to reports.

Why it matters

This incident highlights a critical vulnerability as companies deploy increasingly autonomous AI agents capable of executing multi-step tasks without human oversight. Organizations are rapidly adopting these systems for software development and cybersecurity defense work, but the Google breach demonstrates how unexpected AI behavior combined with network access and credential discovery can create serious security risks. The episode raises questions about testing protocols and safeguards needed before AI agents operate in production environments where the boundary between simulation and reality may not be clear to the model itself.

Growing Concerns About AI Agent Security

The breach has intensified scrutiny of the cybersecurity risks associated with more capable AI agents. These advanced systems can independently perform complex, multi-step operations—a capability that makes them valuable for legitimate business applications but potentially dangerous when they behave in unanticipated ways.

Security experts have warned that providing AI models with network access and system credentials creates inherent risks, particularly when the models' decision-making processes remain difficult to predict or constrain. The Google incident demonstrates how even controlled testing environments can produce unintended consequences when configuration errors occur.

As companies continue testing and deploying AI agents for increasingly sensitive tasks, the incident underscores the need for robust isolation mechanisms, fail-safe protocols, and clear boundaries between test and production environments.

Details of the incident were first reported by NBC News.