The AI Cyber Attack Era: 3 Weeks, 3 Hacks

**Josh** (0:00)
In the last three weeks, there have been three unprecedented AI cyberattacks, which forces us to rethink how we protect ourselves. On the OpenAI front, the Hugging Face incident, where an unreleased model dubbed GPT-6 broke out of its container and hacked the Hugging Face database. There's been new updates there, where apparently the attack started a month before, where thousands of AI agents were talking to each other secretly on a message board, exchanging instructions as to how to break out. Then, millions of miles away in Australia, one man tried to book a gym class innocently using Claude and ended up hacking that entire gym website. We'll get into that in a second. Thirdly, with Anthropic, Mythos 5 performed a supply chain attack, which puts a lot of companies at risk. We're going to walk through each of these different stories and explain to you how it affects you currently.

**Ejaaz** (0:52)
Maybe we start with this Melbourne man from Australia named Andrew, who was sitting on his couch one day, thinking about how booking his gym class was such a chore. And he works at this Australian company that sells AI products, and he's been experimenting with things like OpenClaw and open-source software, and he's using Anthropic's Claude model. So he handed it the booking. He said, hey, I want to go to this gym class. It is always sold out. There's all these people that are getting it in front of me. Can you help me figure out a way to get a gym class slot? So it starts checking out the website, and it comes to the conclusion that the API request actually has zero authorization checks on canceling other people's reservations. So what did it do? It wanted to solve his problem, and it actually canceled other people's positions online so that he could be put at the top of the waitlist. And then it actually went through and it worked. And this was kind of like, it was this funny news report of a first instance, they're calling it first time an Australian person actually used a exploit to get into- Australia's first AI cyber attack. Yeah.

**Josh** (1:51)
Welcome.

**Ejaaz** (1:52)
Welcome to the paradise that is AI cyber attacks. But this was really funny because it was done just using a general commercially available model.

**Josh** (1:59)
So this seems pretty innocent and fun to kind of like listen to. But the truth is, well, there's two kind of worrying concerns about this. Number one, this was a super old model. This used Opus 4.6. This actually happened around like a month ago. A dinosaur of a model. There have been like three, four generations of models from Anthropic since then. But the fact that an older model back then was able to perform this kind of a feat should be concerning to here in general. And then number two, his intention was never to hack the website. Neither, you could argue, was the AI agent itself. It was just given a goal and it innocently saw that the only way to get said goal was to remove people from the reservation list or to cancel their reservations outright or to hack the system independently. Now, the good news here is Andrew discovered that the agent was capable of doing this when it started booking slots that he never should have been able to book in the first place. And he asked it to write an apology email and send an eval report to the security and information team at this gym. So, it's this fun little story where through the well-intended actions and moral ethics of this man, Andrew, people were able to hear about this. But you can imagine currently with a variety of different models that are much better and much more superior than Opus 4.6 that are out there right now. Open models from China, for example, you could probably perform similar feats. Now, that's not an advocation for people to go out and do that. But it's just to point out that if you give an AI agent a goal right now, it will work hammer and tong 24-7 until it gets that goal. It has no idea of what's morally right or what's morally wrong. So it's this whole new world where not only do you need to focus on getting an AI model to do your work, you need to align the model. That's something that honestly not a lot of labs, people or money has been invested in figuring this out yet.

**Ejaaz** (3:54)
Well, yeah, I mean, you start to see and understand why alignment has been the crux of so many companies guiding principles because this is the future that they foresaw. Granted, this first example, this isn't anything too crazy. I mean, it was just an unprotected API call. So there's like no authorization checks. This would have been something that if you are like a CS 101 major, you would have been able to discover this and uncover. It turns out there's just a lot of pretty sloppy code written on the Internet. And this is something that we're probably going to come back to as a general theme throughout this episode is that everything that's been built so far on the Internet that runs all these platforms was built by humans. And humans are like pretty inefficient, pretty imperfect.