A.I.’s Great Escape

**Ian Krietzberg** (0:03)
We got hacked like crazy. We've never seen anything like this before. We were able to prevent it. We used AI systems to contain it. We're okay, like nothing terrible happened, but holy shit, this one was crazy.

**Peter Hamby** (0:20)
Welcome to The Powers That Be Daily, Puck's podcast focused on the intersection of Wall Street, Washington, Silicon Valley, and Hollywood, and the players who run it all. I'm Peter Hamby. It's Tuesday, August 11th. Today, I'm joined by Ian Krietzberg, who fills me in on a cascade of security screw-ups at the big AI companies, even though they keep asking the public to just trust them, as they build even more powerful models. The Trump administration has announced it will review their most advanced models with a new AI framework. But as Ian explains, we don't know what's in the White House policy or whether it has any teeth as we race toward AI singularity.
We'll discuss all that and much more on today's episode, The Powers That Be.
Happy Tuesday, everybody, and welcome to The Powers That Be. It's your boy Peter back from a little vacation. I went to Maui. What a treat. I can't believe Hawaii is actually a state. Every time I go there, I am in a surplus of calories from all the pina coladas. We drank. Thanks to Dylan. Thanks to Leanne for covering for me on this podcast when I was gone, and the team for letting me take a break from filing, as we like to say. I'm joined today by my colleague, Ian Krietzberg, who's going to update us on the latest round of security incidents and worries with big AI companies who keep asking us to just trust them, even though the evidence points in another direction. Ian's also going to tell us about some news out of the Trump administration. We've got a new kind of vague framework for government security testing on AI frontier models. We don't know a lot about it because they're not releasing the details of it to the public. It is classified. But it does sort of indicate, as Ian and I have been talking about on this podcast, that the Trump administration, while certainly not a regulatory state in any way when it comes to big tech and AI, is winking at some of these security concerns and at least having a conversation about it. We'll get to the Trump administration's framework in a moment. But first, Ian, good to see you. Can you catch us up on some of these more recent AI incidents that have happened? Before I went on vacation, of course, the big one was OpenAI's model escaping their sandbox, hacking into Hugging Face. It looks like some other companies. But Ian, it sounds like other big AI companies, Meta, Anthropic, have also found some security loopholes. What's been happening?

**Ian Krietzberg** (3:01)
Yeah, while you were hanging out in Maui, a lot of stuff has been going on my side of the industry. It's been a little crazy. And this is all, I mean, everything up until Mythos, which we've talked about many times, right? Mythos was kind of like the moment that things changed. And we've been living in that post-Mythos world where things get crazier every day.
And we can expect, I think they will probably continue to do so. So, where we lost off last was, yeah, this OpenAI Hugging Face incident, which is complicated because we don't really know exactly what happened, right? We know that we were in an evaluatory environment, right? And they were evaluating for cybersecurity capabilities, and so they take off the guardrails in order to do that.
They prompt it to, you know, showcase its abilities very basically, right? It's complete this kind of cyber benchmark test. And then, it's not clear exactly what happened next, right? Clearly, vulnerabilities were exploited, companies were hacked, security around the evaluation procedure was very much not hardened by any means. And so this turns into a question that everyone has been debating since, which is, should we be more concerned about security that's not good or a very capable prisoner that needs securing in the first place? How much can we secure these systems? How much did we do? These are all the big questions. These are the same questions today, by the way. And it's still not very clear. But what that incident did was it prompted everyone, if you take their word for it, to kind of go back and look through their logs, understanding that this is a thing that could happen. That unless you're monitoring the activities of your systems very, very, very, very closely, they might do things because agentic systems are action-taking systems. And as cybersecurity experts have been warning, since they first started talking about agents and computer use and these types of things, things will go wrong when you give an algorithm agency to operate autonomously.