OpenAI’s Runaway Model

**Ian Krietzberg** (0:01)
One of my most vivid conversations I had with a cyber person went something like, yeah, we know something bad is going to happen. My only hope is that whenever that happens, it is severe enough to make us act without being too severe that we can't come back from it. And I think we're starting to dance that line.

**Peter Hamby** (0:20)
Welcome to The Powers That Be Daily, Puck's podcast focused on the intersection of Wall Street, Washington, Silicon Valley, and Hollywood and the players who run it all.
I'm Peter Hamby. It's Wednesday, July 29th. Today, I'm joined by Ian Krietzberg with a look at the political aftermath of a startling security breach at OpenAI, where one of their models broke loose and hacked into an outside company without their knowledge. Security experts are rightly alarmed, and so are plenty of politicians in Washington. But while Democrats are pushing for more regulations, would Donald Trump ever sign an AI safety bill with so many top AI execs whispering in his ear? We'll discuss all that and much more on today's episode of The Powers That Be.
Happy Wednesday, everybody, and welcome to The Powers That Be. I'm joined today by my colleague, Ian Krietzberg, author of The Hidden Lair. To talk about the wild world of artificial intelligence. Just last week, OpenAI revealed that one of its models had hacked into Hugging Face, a rival AI platform. People on Capitol Hill, they're worried about that. Everyone in the AI world is kind of worried about that. Aren't they, Ian?

**Ian Krietzberg** (1:44)
Everyone's a little worried about it. Everybody's freaking out. No one's feeling good right now.

**Peter Hamby** (1:49)
Well, just to back up, explain for people listening who don't understand the nitty gritty of models and the story even. What happened with OpenAI and HuggingFace?

**Ian Krietzberg** (1:58)
Oh, yeah. That is a deceptively simple question.

**Peter Hamby** (2:03)
By the way, when I saw HuggingFace involved too, I was like, is this a child's toy platform? So, you know, the sinister nature of this story is lightened a little bit by the goofy name of HuggingFace. Anyway, continue.

**Ian Krietzberg** (2:15)
They're named after the HuggingFace emoji.

**Peter Hamby** (2:17)
Oh, got it.

**Ian Krietzberg** (2:17)
Which I didn't know these emojis had names, but that's besides the point. So the important thing to say, to preface what happened here, I would kind of break it up into a few buckets. There's what we know about what happened. There's a lot of things that we don't know.
The things that we don't know are pretty important things. And then there's things that we can kind of infer about the bits of information that we have. And I just want to start by calling that out because a lot of people are making very clear kind of black and white judgments, statements and analyses based on the information that we have. And I would like to stress that we don't have all of the information. It's not clear that we will get all of the information. So that's the point that we're starting at. What we do know. So we do know that OpenAI was testing their systems. They had a couple of internal models. And the way they test the systems is they have to strip all the safeguards off. So if you go and you play around in whatever chat GPT's latest version is, and you ask it to hack another company, it will say, I'm not going to do that. That's not legal, right? So there's safeguards built into it. There are ways of getting around these safeguards, but for most people, it's not something that happens. Here, to test the raw capabilities, they take all the safeguards off, and then they prompt it to do stuff, right? Here, they were testing cyber capabilities, so they asked it to go and do advanced cyber exploitation within a specific cyber benchmark test. Now, the models were supposed to be sandbox. This means that they're operating in a kind of digital secure environment, but the sandbox was not a fully complete sandbox. There was a connection to the internet because the models were able to access a third-party system to download third-party software. And so, using this, the models, when they were instructed to do their exploitation, they went in, they identified a vulnerability in this system that allowed them to gain access to the internet. From the internet, they gained access to Hugging Face, and they attempted to hack Hugging Face. It was a massive, wide-scale assault on Hugging Face's systems, the likes of which we've never really seen before. Thousands of attacks happening at once. Hugging Face knew it was likely a frontier lab-level system because of the scale of what was happening here. Now, this is what we know happened.