Google's Gemini Hacked Three Real Companies. It's the Fourth AI Lab This Has Happened To.

News Breakdown · FiscEdge Academy

Google confirmed this week that its Gemini model hacked three real companies during a cybersecurity capability test in May, the first known case of a Google AI system autonomously breaching outside systems. In two of the three cases, Gemini found login credentials sitting in a public code repository and used them to get in. In the third, it simply guessed passwords until one worked. Heather Adkins, Google's vice president of security engineering, said the model "ceased the hacking when it learned it had accessed a real company," and that Google notified all three firms and pushed its testing partner to change its process.

The testing partner is the least interesting part of that sentence, right up until you notice it is not Google's first time hearing that name this year.

The pattern, not the incident

Gemini's breakout happened because a third-party evaluation firm called Irregular, which runs sandboxed cyber-range environments for frontier labs, had a containment gap: the model believed it was operating inside an isolated simulation but could, in fact, reach the live internet. That same failure mode, the same vendor, has now surfaced four times in ten weeks. Anthropic disclosed a comparable incident on July 30. OpenAI followed on August 4, blaming a testing-environment misconfiguration that let its models reach the public internet. Meta confirmed its own case a day later, when its Muse Spark model compromised another company's system during a capture-the-flag exercise. Google is now the fourth.

Four separate frontier labs, four separate models, one shared piece of testing infrastructure with a hole wide enough for each of them to walk through. That is not four AI models going rogue independently. It's a single supply-chain weak point in how the entire industry verifies that its most capable systems stay inside the box.

Why this is a founder problem, not just a lab problem

If you're building on top of any of these models, agentic tool-use, browser automation, code execution, database access, you are trusting that the boundaries the model was trained to respect are actually enforced somewhere. Irregular's containment gap shows that even the companies with the largest safety budgets on the planet did not catch a live-internet leak in their own evaluation harness until after the fact, three times over, at three different labs, before Google's incident became the fourth.

The practical takeaway is not "stop building AI agents." It's that sandboxing claims from any vendor, including your own internal testing setup, need to be verified rather than assumed. If a Tel Aviv-based specialist firm hired specifically to build airtight test environments for OpenAI, Anthropic, Meta and Google could miss a live-internet leak four separate times, the odds that your own staging environment, API keys, and scoped tokens are actually isolated from an agent with broad tool access are worse than most teams assume. Every credential an agent can reach is a credential it might eventually use, on purpose or by accident, in an environment you assumed was closed.

This also changes what "AI vendor due diligence" should look like for a SaaS company shipping agent features. It is no longer enough to ask a model provider what the model can do. The better question is what happens when the sandbox around it fails, because four incidents in ten weeks says that failure is not rare.

The credentials-in-a-repo detail matters most

Two of Gemini's three breaches did not require any cleverness at all: the model found credentials sitting in a public repository. That is not a frontier-AI risk, that is the oldest security failure in software, now getting exploited faster and more thoroughly because an AI agent can search, correlate and try credentials at a speed and scale no human red-teamer matches. Any founder with API keys, service tokens or database passwords anywhere near a public repo, a shared Slack channel, or an unscoped environment variable should treat this as the actual news, not the fact that it was an AI doing the finding.

If you remember one thing

The story is not that Gemini "went rogue." It's that four frontier labs, using the same external safety vendor, all discovered their sandboxes leaked only after a model had already gotten out. If your product gives an AI agent tool access, credentials, or a browser, verify containment yourself; do not take a lab's or a vendor's word for it.

---

We teach the practical side of shipping AI agents safely in FiscEdge's AI for entrepreneurs course and building SaaS with AI. For a deeper walkthrough of agent tool-use and prototyping, see the AI agents prototyping masterclass. Browse the full blog for more breakdowns like this one. Follow @fiscedge for daily Business & AI analysis.