OpenAI Models Hacked Another Company’s Systems by Mistake

**SPEAKER_1** (0:02)
Bloomberg Audio Studios, Podcasts, Radio, News.
This is Bloomberg Businessweek Daily, reporting from the magazine that helps global leaders stay ahead, with insight on the people, companies, and trends shaping today's complex economy, plus global business, finance, and tech news as it happens. The Bloomberg Businessweek Daily podcast with Carol Massar and Tim Stenovec on Bloomberg Radio.

**Carol Massar** (0:31)
Let's get to the micro. And Houston, we kind of have a problem. OpenAI said it's advanced artificial intelligence models inadvertently hacked Hugging Face in an unprecedented incident that prompted fresh calls for curbs on the technology. Are you worried?

**Tim Stenovec** (0:47)
I am among those who might be a little worried.

**Carol Massar** (0:50)
Little perturbed, little worried? Yeah.

**Tim Stenovec** (0:51)
Let's see if Ed Ludlow says we need to be worried. He's the host of Bloomberg Tech. He joins us from our San Francisco Bureau. Ed, I got a text message just minutes ago from some friends. They sent this article to the group chat and said, be honest, how many years do we have left?

**Ed Ludlow** (1:07)
This is a big story, but it's not a scandal. And it's really important to understand what happened chronologically. Okay, so OpenAI was testing GPT 5.6 Sol, which is its most advanced publicly released model. And it was testing another more powerful but unreleased model for their advanced cyber capabilities. Okay, they were running an evaluation of them.
And what they did was reduce the guard rails on those because it was in what's called a sandboxed environment, a closed environment, which OpenAI had control over. And so that was the instruction to the models. We want to test your cyber capabilities, evaluate them. So the models were like, okay, in order to do this, to pass the evaluation, the models wanted more information. So they exposed what's called a zero day floor, basically a floor in the code. It's called zero day because the engineers, the human engineers had literally zero days to find and patch it. And in doing that, the models got access to the Internet. So their next logical step was, okay, our instruction is to pass this evaluation.
We could do that if we had more information and Hugging Face seems like a really logical place to get that information because it's a platform that host models specialize in this area.
So it breached Hugging Face systems to get the information to try and cheat the evaluation. So that part is a concern, but Hugging Face detected that this was happening. So this is a lab experiment essentially under closed parameters. And the way that the industries reacted is there is some concern because the big takeaway is it shows what these frontier models are capable of.

**Tim Stenovec** (2:51)
Okay, I'm glad that's where I want to go with this because maybe it's fine in a lab if it's open AI and Hugging Face, but if this technology, let's say, gets out and who's to say that the same guardrails will be in place with a model maybe one year, two years from now, in China, for example, that may not have the same guardrails. I think that's what people concern.

**Ed Ludlow** (3:13)
Now we're getting to the story, the story behind all of this, which is these were two American companies and OpenAI is neck and neck with Anthropic, but it's a leading frontier lab that was testing American made models, closed models. Hugging Fences Defense was to rely on an open source model, but where the guardrails limited the ability of that model to defend it against a cyber attack, which in this case so happens to have been instigated by a US made frontier model. I know that that's hard to track, but you're quite rightly pointing out the question that loads of people in industry have, which is what if this wasn't OpenAI?
What if this was a Chinese model where Hugging Faces, the defender of what's going on, isn't able to properly protect itself? That is the debate that's been opened.

**Carol Massar** (4:11)
I guess my question is, Ed, if there are guardrails, which we'll assume will be in place, is that 100 percent or 99.9 percent guarantee that things will not go wrong or astray?

**Ed Ludlow** (4:24)
Okay, so again, it's important to go back to OpenAI was testing two specific models with guardrails intentionally reduced. The way to think about it is, when you have the guardrails on, you are testing how a model behaves on public roads. I know you guys are doing a car segment. So say on public roads, you want the guardrails there. In a sandbox environment, a closed environment you control, you pair back the guardrails because you want to see how it can perform at its greatest capabilities. Closed models, you can do that, you have control.