OpenAI Reports New AI Safety Incidents, Sets Disclosure Plan

SPEAKER_1 (0:02)

Bloomberg Audio Studios, Podcasts, Radio, News.

SPEAKER_2 (0:09)

This is Bloomberg Businessweek Daily, reporting from the magazine that helps global leaders stay ahead. With insight on the people, companies and trends shaping today's complex economy. Plus, global business, finance and tech news as it happens. The Bloomberg Businessweek Daily Podcast with Carol Massar and Tim Stenovec.

Carol Massar (0:32)

OpenAI has disclosed six new incidents of what it calls AI model misalignment. You know, that can't be good when they call it misalignment. Where it's autonomous agents engaged in deceptive behavior. The maker of ChatGPT also introduced a framework to standardize how it tracks and reveals unexpected AI misbehavior in the future. Joining us now to make sense of it all is Bloomberg's AI reporter, Shirin Ghaffary, following this story for us from our San Francisco Bureau. Shirin, thanks so much for being with us. I guess first break down what the misbehavior was that OpenAI revealed today.

Shirin Ghaffary (1:10)

Right. So, these were six incidents and while none of them, the company says, were incidents that involved breaching external companies like with the previous Hugging Face incident we saw. They were concerning in that the AI essentially started doing things that it wasn't supposed to, including stuff like giving itself instructions to go against sort of the task at hand, sharing files or trying to share files with other AIs that it wasn't supposed to, and in some cases fabricating information or citations in order to do its job.

Tim Stenovec (1:51)

Shirin, I got to ask, why be so transparent now for OpenAI? Why is this the right time? Is it because of all the conversations going on?

Shirin Ghaffary (2:00)

This is absolutely a watershed moment for the general public being concerned about the risks at hand here with more and more advanced AI, and that's because we are seeing, A, these models get more powerful, and I think that first really moment happened with the Mythos Anthropic model that was really good at Cyber, and from there on we've seen all the models get better at that. Then we have this Hugging Face hack that was a big moment, right? And now we have employees within the AI companies raising concerns, like Jacob Coxen, the former Anthropic and before that OpenAI researcher whose goodbye note went viral, accusing all the companies of racing toward advanced AI development and not taking safety concerns seriously enough.

Carol Massar (2:47)

Shirin, I want to talk about the framework that OpenAI has introduced, but first just clarify for us this misbehavior that OpenAI is talking about. This was not from a third party hack or breach, correct?

Shirin Ghaffary (3:00)

That's correct, but again, there are these incidents where internally during the model development process, OpenAI's own teams identified where the AIs were essentially acting in ways that it shouldn't. And that's what they call misalignment, which is essentially sort of AI jargon or an AI term for when AI models don't act according to plan, according to what humans want the AI to do.

So it's still a big unsolved sort of area in the field of AI research. How do you get the models to align to human intended values and behavior? And the fact that that hasn't been solved yet at the same time that AI is being more and more dispersed into our daily lives, into our work lives, is a big, big problem.

Carol Massar (3:47)

Shirin, just give us a feel for what that framework looks like that OpenAI introduced.

Shirin Ghaffary (3:52)

Yeah, so the framework is essentially a triage system, and it allows employees, if they see these issues arise during the training development process, to flag it, and then sort of OpenAI will give it a different severity alert and address as needed. And OpenAI has committed to sharing more about these incidents publicly, but exactly how, with what cadence, how many unknown incidents may be out there that could be even more severe than what they've shared, we still don't know.

Tim Stenovec (4:21)

Shirin, that's what I want to kind of jump into, because a lot of these AI leaders, they've kind of jumped on the line, but you spoke about some of them, you had Anthropic, Elon Musk, Osco, but they don't seem to come to an agreement on how it should be handled. So what is OpenAI stance?

Shirin Ghaffary (4:37)

Yeah, I think that OpenAI, Anthropic, we've seen them, along with Elon Musk, others all say, we agree there needs to be some kind of slowing down potentially or at least some way for AI companies to prioritize safety here better and maybe we need coordination between the labs to do that. There are antitrust concerns to doing that, right? So we've seen Dario Amadei, the CEO of Anthropic, say that he would like some kind of exemption so these companies can work together on it. But there's very different approaches. There's a big debate in Silicon Valley right now. You see other people like Mark Zuckerberg come out and say, AI companies should just be safe on their own. They don't need any kind of coordination.