OpenAI's rogue agent and Silicon Valley’s China problem

**Danny Fortson** (0:05)
Hello, and welcome to The Times Tech Podcast. I am Danny Fortson, back in Silicon Valley, no longer in the city of London.

**Katie Prescott** (0:11)
Back where you belong. And I'm Katie Prescott, here, still here, boringly, in the city of London, where I'm covering all things tech for The Times.
Danny, we speak to lots of entrepreneurs on this podcast who do weird and wonderful things. And that's one of the most amazing things about covering tech. I don't know the true joys, but I think today's guest might be the quirkiest person that we've ever spoke to, possibly.

**Danny Fortson** (0:37)
Certainly the quirkiest idea.

**Katie Prescott** (0:38)
Certainly the quirkiest idea. He has a business called MeatLayer, which does what it says on the tin. It provides a MeatLayer, also known as a human being, for AI bots to rent.

**Danny Fortson** (0:51)
Yes, indeed. The premise is that AI agents can hire humans to do jobs that they can't, like installing shelves, delivering flowers, because of course, AI agents are very, very powerful, very capable, like our friend Norman.
Drawback, no arms, no legs, no driver's license. So that's the idea. Lots of questions.

**Katie Prescott** (1:12)
Many, many questions.

**Danny Fortson** (1:14)
The agents go rogue or whatever it may be.

**Katie Prescott** (1:16)
And I think it also plays pretty nicely into one of the biggest stories in Tech of the Week, talking of agents going rogue, which was some news that OpenAI put out about one of its agents going quite scarily rogue.

**Danny Fortson** (1:29)
Yeah, very, very, very, very big story out here on my side of the pond. It was basically a training exercise to see how powerful this, the cyber abilities of these, of these models were. And it basically broke out of its training environment, accessed the open internet, and then went to another company and broke in there.

**Katie Prescott** (1:52)
And it went to Hugging Face, right? Because that is where developers' code and AI models are hosted. And so it thought it might find the answer to the challenge that it was doing in this training program on Hugging Face.

**Danny Fortson** (2:08)
Yeah. But there's another story we need to get into, which is the release of Kimi K3, a super-powerful Chinese AI model, which people are saying could, you know, it's certainly threatening America's supposed tech supremacy when it comes to frontier.

**Katie Prescott** (2:23)
Yeah, we're going to be talking about why this has caused such an enormous storm in the tech world and what it tells us about the AI arms race.
Well, let's get into it and start with that OpenAI bot cyber attack, which is totally wild and it feels like everything that everybody has been scared about, bots going rogue and attacking other companies has come to pass. And so it's a really intriguing story. So as you were saying, this bot was taking part in a training program, within OpenAI, but it broke out.

**Danny Fortson** (2:57)
Yeah. So the idea was this is like an exercise. So at OpenAI, you set up what they call a sandbox, which is a secure environment, and then you can test your models to, their various capabilities to do different things. This one wants to test its hacking capabilities. And it was two models. It was Sol, the latest model, and then as yet unreleased model working together. And what it did was those two together, figured out a way to break out of the sandbox, get on to the internet, then break into Hugging Face, and try to exploit there, because the whole goal, which is why people are freaked out, is like you give these systems a goal, and then see if they can achieve it. But the thing that surprised them was that, one, these are two agents operating entirely autonomously, and doing whatever they could to achieve their goal. And the reason they went, as you said, to Hugging Face is because it's this repository of tons of AI code, and they figured like, if we go there, this might be a way to like, maximize our success in the given task. And so Hugging Face freaked out, and this is the other piece of the story, which is so fascinating, which leads right to what we're gonna talk about next with Kimi. Hugging Face sees like, oh, their systems are being attacked. They're like, uh-oh, this is a problem. They start using different, they didn't say which AI models, but US AI models from the Frontier Labs. But they wouldn't allow them to basically do the work to stop the bots from breaking in and causing havoc, because those AI models from the US companies have certain guardrails that basically disallow it, prohibit it from doing things that sound like a hack.