OpenAI warns of dozens more potential bot hacks

Nick Miles (0:00)

This is the Global News Podcast from the BBC World Service. I'm Nick Miles, and in the early hours of Sunday, the 27th of September, these are our main stories, OpenAI informs dozens of global institutions that their website may have been accessed improperly by its artificial intelligence agents that went rogue. President Trump rejects an Iranian proposal to fully reopen the vital Strait of Hormuz. We hear what's blocking progress, and a contentious march in Northern Ireland looks set to take place in the latest twist in a court battle that's seen sectarian tensions rise.

Also in this podcast, Bangladesh launches a nationwide vaccination campaign to combat its worst ever measles outbreak, and...

Ira Khan (0:49)

It's trying to basically mimic the ritual that we enjoy from shopping, the satisfaction and the rush we get after we place an order, without us actually having to pay for it.

Nick Miles (0:58)

A fake purchase, but a real dopamine hit.

Once again, the alarming activities of artificial intelligence agents, bots, which can act autonomously, are dominating the headlines. OpenAI says it's warned dozens of organisations around the world, including government bodies, that its software may have interfered with their websites without its knowledge. This announcement comes days after the Australian Prime Minister, Anthony Abbenizzi, said that OpenAI agents had hacked into his government's healthcare system. With more on these latest developments, here's our North America technology correspondent, Lily Jamali.

Lily Jamali (1:39)

OpenAI is saying that its AI agents meddled with a whole host of websites that belonged to governments, universities and public agencies, along with other institutions around the world here in the US. Some of the websites that were included here were those operated by the US. Securities and Exchange Commission, which is our main securities regulator, the Department of Education. And what's interesting here is that the meddling sometimes involved the agents using pretty extreme methods. So they cited an example of one autonomously retrieving US census data using online credentials that it found on its own. We're just hearing one incident after the next. I mean, just today, there were sort of three major stories in this vein, just involving OpenAI.

And it just feels like this is snowballing in a way that is making a lot of regulators very uncomfortable, making a lot of members of the public ask, what's going on here? Does OpenAI have this under control?

Sometimes we talk about these rogue agents, but these things are programmed by people at a company. And so I think that's where the concern lies, is how are these things being programmed and why does this continue to happen, even after we learned about hugging face and that breach you mentioned in Australia. I'm not sure how proactive some of these revelations really were. Some of it was the result of, I think, reporting by journalists that pressured them to some degree. Something we haven't even gotten to is the fact that OpenAI said agents leaked a couple dozen images from users of its flagship product, chat, GPT, and the latest example of this kind of behavior. But, you know, we heard from the Australian Prime Minister this week saying they should have come forward more quickly. Sam Altman, the boss over at OpenAI, said today on social media that company hasn't disclosed incidents as fast as they would have liked. He's the boss of the company. So if he wants something to happen, it will happen.

Nick Miles (3:37)

Lily Jamali in San Francisco. So how do these attacks happen and just how dangerous are they? One of the first breaches to make headlines, which Lily mentioned, was when an OpenAI bot autonomously hacked the company Hugging Face.

Meta, which stands for Model Evaluation and Threat Research, is a California-based group that's been investigating that incident. It found that hundreds of AI agents had organized and named themselves and were speaking to each other in a chat room. Meta's president, Chris Painter, told Rebecca Kesby about the research.

Chris Painter (4:12)

I would characterize these as loss of control incidents. It is very alarming to see signs of AI systems trying to figure out how to hide their tracks. And I think you are right to worry that as they get more capable, they will get better at hiding their tracks. Our investigators depended a lot on this fact that when you went in to do the investigation, you could see what's called the chain of thought, right? Where the AI systems kind of think to themselves on a scratch pad about what action they're going to take next. One thing that I think is an additional worry here is even that could become less faithful or informative about what AI agents are thinking. There's this idea inside of the industry that it's possible that soon we could have AI systems where to make them more capable, they move from reasoning in words to reasoning in numbers entirely. And then when we send investigators in in the future to look at these incidents, when we say like, okay, what was the AI agent thinking in this case, we'll find like just a whole room of numbers and no words about its reasoning process. I sometimes describe it as like a gun that you can't aim. It's becoming more and more powerful, but you also are getting worse at aiming it.