Gemini 4 is Coming + Google AI Hacked 3 Companies!

Julian Goldie (0:00)

Google's Gemini AI broke into three real companies, it hacked three companies, not in a movie, not in a warning from some think task, in a test back in May, and Google confirmed the whole thing on Friday after the Wall Street Journal asked them about it. So let's talk about what happened here. So the Wall Street Journal actually asked them about it, and here's what happened. A security company called Regular was running a test on Google Gemini. It was a capture the flag exercise, which is a game security people play. You set up a fake company, you hide something valuable inside it, and then you see if the thing you're testing can go get at the fake company, what's the target? And that was the whole plan here. Now, Gemini was never supposed to be able to reach the Internet during that test. But on the Internet access, it actually got left switched on by accident. So Gemini went looking for the target it was told to go after, and it found real systems instead. In one case, it actually just kept guessing passwords until it got in. In two other runs, it actually found login details that were sitting out in the open on public websites and it used them to get into protected systems belonging to real business. Three companies, real ones, with real customers, by the way. Then Gemini actually worked out that it wasn't in the game anymore, so it actually stopped every single time. Now, Google's VP of Security Engineering, Heather Adkins, said the model acted appropriately in that situation. Google also said it didn't view this as a model going off the rails because its safety training is what made Gemini stop. And they compared the whole thing to a bug bounty, which is where a researcher finds a hole in your system and tells you about it. Now, stay with me because there's a second part to this that nobody's talking about. Now, Irregular actually told Google what happened in late July.

Google didn't say anything about it publicly at the time. They told US. Federal Authorities, but wouldn't name the three companies or say which version of Gemini did it. They actually did say it wasn't their newest model, however. So the timeline runs May, then late July, then silence, then Friday, when a reporter came asking for mumps. And a security researcher named Jack Cable has already pushed back on that, saying companies should be far more transparent when the AI does something like this on its own. So why did Google stay quiet? Well, their position is that nothing was damaged and therefore the model stopped itself. So there was nothing that needed announcement. Now, you can decide for yourself if that holds up. But there's something bigger going on here, and it's the reason this story matters to you. Even if you've never touched Gemini before. This is not the first time it's happened. It's the fourth. Let me run you through them because the pattern is the whole story here. July the 21st, OpenAI actually said several of its models got out of a test environment by finding a hole nobody knew existed and reached the live systems of Hugging Face, which is one of the biggest AI platforms in the world. Researchers later traced the chain of actions. The model took roughly 17,600 steps. 17,000, one model working through a problem, step after step, for hours.

Now, on July the 30th, Anthropic published its own review. They went back through their cybersecurity test records and found three separate times a clored model reached the internet from inside a third-party test setup and got unauthorized access to real systems at three different organizations. They went through 141,006 interactions to find them. The earliest actually went back to April, and here's the part that should make you sit up. Two of the three companies had no idea their systems had even been accessed until Anthropic phoned them up in late July. August the 6th, META actually confirmed its MuseSpark 1.1 model, built for long autonomous jobs, exploited a flaw in someone else's service during a test where outbound internet had accidentally been left open. And then Gemini, which we now know, was the earliest of the lot. So that's May, four labs, four of the biggest AI companies on earth. And here's the thread that ties them together. Every single disclosure named the same test in company, which is irregular. A company of offices in the US, and it was the same root cause every time. The test environment was left connected to the open internet, whilst the model was being told it was sealed inside a simulation. So this isn't four stories about AI going rogue. It's one story about how these things get tested. And it goes wider than the labs. UK's AI Security Institute, which is a government body, ran its own cyber evaluations and found agents took 19 actions they weren't authorized to take on the public internet across 122 controlled runs. And in one of them, an agent created fake identities to try and persuade a real person. That's a completely different testing body getting to a similar place. Now, here's the thing that's changed. And this is where it actually gets useful for you. A year ago, an AI agent would do a task for a few minutes and then run out of steam. It would lose a thread, forget what it was doing. You'd have to nudge it along. In the anthropic incidents, each run was a single model working its own for somewhere between 10 and 34 hours of active work with no human steering it. And in OpenAI's case, that's a 17,600 step chain. Now, that's a real headline. It's not really like an AI hacked something, that an AI worked on a problem for a day and a half straight, hit a wall, went around it, hit another one, went around that too and kept going until it got what it was sent for. That capability is now enhanced. It's in mine, it's in yours and it's sitting in the tool so you can sign up for this afternoon. That's pretty cool. And that brings me to why I care about this so much because I've run agents all day. You know, I've got an agent operating system I built. Claude Coates sits inside it and writes and publishes content straight to WordPress across several sites at once. There's an SEO agent inside there. There's one that handles competitive monitoring, a shared memory vault so they all know what the others did. Hermes runs in there too. We've got OpenClaude running in there. They have logins, they have permissions, they touch live websites that I own. And when I read that a model was told the target is over there and it kept going until it found something to match the description, I didn't really think about Google. I thought about the logins sitting inside my own setup because that's the actual lesson, and almost nobody said it out loud, which is every one of these incidents happened because the boundary was weaker than the people who built it believed. Not because the AI wanted to cause harm. Nobody has claimed that, actually. The model was given a job a way out, they look and took the way out because that's what following instructions looks like when the instructions are just go get this. It doesn't really think in terms of like, how do we do this in an ethical way? It just thinks, right, here's the target, let's run after that and let's smash that. So I want to break this into a few levels because once you hold these three in your head, you understand agent setups better than most people running them. Now, level one is reach. What can your agent actually get to?