OpenAI’s Escape Room

Ian Krietzberg (0:01)

The agents have exploited their sandboxed environments, which means they weren't actually sandboxed, to improvise a messaging board so they could communicate with each other. And they banded together to do things such as hack Hugging Face.

Peter Hamby (0:20)

Welcome to The Powers That Be Daily, Pucks podcast focused on the intersection of Wall Street, Washington, Silicon Valley and Hollywood, and the players who run it all. I'm Peter Hamby. It's Tuesday, September 8th. Today, I'm joined by Ian Krietzberg, who tells me about ongoing concerns about OpenAI's Hugging Face incident. As we learn that the company's autonomous agents apparently understand the concept of collective action and sacrifice. Ian and I discuss what we know and what we still don't know about Hugging Face and other security incidents involving the company's models.

We also talk about Bernie Sanders introducing a bill to ban superintelligence as he plants a flag firmly on the populist doomer side of the AI political debate.

We'll discuss all that and much more on today's episode of The Powers That Be.

Happy Tuesday, everybody, and welcome to The Powers That Be. I'm joined today by my colleague, Ian Krietzberg, who has been covering the ongoing fallout from OpenAI's rogue agents, the Hugging Face fiasco. It turns out, shocker, there was a lot we didn't know at the time, and we keep learning more, and some of it is kind of frightening. We're also going to talk about Bernie Sanders, who has been one of the biggest crusaders on Capitol Hill against the growing power of AI companies. He wants to introduce a Bill Banning superintelligence. I want to talk to Ian about that. One programming note, when you're done listening to The Powers That Be today, go subscribe to The Making of a President, my brand new podcast with Puck about the 2028 presidential race. Not the horse race, not the headlines, not the Trump push alerts. We're trying to dig in to how campaigns work. And I'll be spending, gosh, the next two years, hopefully, talking to insiders, decision makers, creative people, inside campaigns, outside campaigns, who are dealing with an unprecedented presidential race. This is going to be a crazy one. So I'm very excited about it. Go subscribe. Anyway, back to Ian. Good to see you, man. I hope you had a great Labor Day weekend. Let's talk about Hugging Face. I remember, gosh, must have been a month ago or something, OpenAI put out a statement saying their model had escaped a sandbox, hacked into Hugging Face. They didn't say nothing to see here, but they kind of meant to suggest, as they often do, that this is just one little mistake, we're figuring it out, and we'll keep you posted. It turns out there's some long tail effects here that we learn about every single day, and you've been digging into it. What do you think has been the biggest storyline in recent days around OpenAI and the post-mortem of the Hugging Face attack?

Ian Krietzberg (3:13)

Yeah, that's a good question because there's probably a dozen of them. But the one I would call out is, so you and I have been talking about this Hugging Face incident all summer, as has everybody in this industry.

This has been described as a pretty big moment for a lot of people in AI. We're seeing a kind of tone shift with people in the AI industry, and we're also seeing this tone shift echoed on Capitol Hill as well. So this is happening in both places. And after this event occurred, there's all this different kind of verbage to describe what happened here, but we'll call it an event. OpenAI promised to do this post-mortem, investigating into what actually happened, which seems like a good thing to do. As part of that promise, they said, hey, METER and Redwood Research, these are two AI safety non-profit research organizations. They said, you guys, you can come in and you can do an independent investigation. We're going to volunteer to let you guys do this independent investigation so that you can tell the public what happened, so it's not just us telling them what happened.

Peter Hamby (4:14)

Have they done something like that before? That seems actually like a good step.

Ian Krietzberg (4:17)

It seems like a good step. Yeah. So OpenAI has had a relationship with METER. They do a lot of safety evaluations. There will be an aspect of almost every model release for the past few model releases. I don't remember exactly how far back it goes, where a component of OpenAI system card and GPT, whatever, will be a METER safety evaluation. So they've had a long running relationship. So they were a pretty obvious party to turn to here, but it bears repeating here, not a government entity.