Who pays when AI agents go rogue?

Katie Prescott (0:02)

Danny, there have been several big virtual queues in the UK in the last week. Number one over the weekend was Glastonbury, which sold out in 42 minutes, I believe. And number two is the British Museum's members sale of buyer tapestry tickets for next year.

Danny Fortson (0:24)

I'm sorry, what, what?

Katie Prescott (0:26)

The buyer tapestry, which depicts William the Conqueror's conquest of Britain in 1066

Danny Fortson (0:32)

That's my favorite tapestry.

Katie Prescott (0:34)

I know, same, it was not actually a tapestry, but anyway, we can get to, is in the British Museum. And it's really hard to get tickets, it's gold dust. And I got Bridget, my instinct agent, to sit in the queue today to get tickets.

Danny Fortson (0:51)

Did she do it?

Katie Prescott (0:52)

No, but not her fault.

Danny Fortson (0:54)

Lazy.

Katie Prescott (0:55)

The website stopped working today at various points, and it happened to my colleague, who's a real person, not an agent.

Danny Fortson (1:01)

You know what this is leading us toward? You need a humanoid robot to stand in line overnight. He doesn't be like, be boo, bop, be be be, I am Katie Prescott. I can't wait to see this piece of cloth.

Katie Prescott (1:12)

Small little piece of cloth. We can have this conversation another time. I'll send you some reading material. We've covered it extensively for Times.

Danny Fortson (1:19)

Well, this is all very important, but shall we get to the pod now?

Katie Prescott (1:21)

Let's move further up the queue.

Danny Fortson (1:23)

Let's move on.

Katie Prescott (1:24)

Oh, my God.

Danny Fortson (1:24)

Yes. Hello, and welcome to Times Tech, where we unpack how technology is reshaping our everyday lives. I am Danny Fortson.

Katie Prescott (1:34)

And I'm Katie Prescott. And this week, we're asking who should be held responsible when AI goes really, really wrong.

Danny Fortson (1:42)

Yeah, so AI companies are currently investigating. Feels like we hear about a new one every day. A number of situations where agents have gone rogue and done things which would be considered crimes if they were committed by a carbon-based life form, a human.

Things like hacking government websites, for example.

Katie Prescott (2:01)

Yeah, this is something I've been looking at for my column, because as we know, when something goes wrong, generally, the only people who benefit are the lawyers. And Danny, you might be in Silicon Valley where the tech's being made, but here in the city of London, I can tell you some of the world's top lawyers are rubbing their hands over this.

Danny Fortson (2:23)

I can imagine.

Katie Prescott (2:23)

And also here in London, of course, we have got the world's biggest insurance market, which is also looking at this and going, hmm, not least because in Anthropics recent IPO prospectus, they were saying, please give us your money, but look at all of these things which could potentially go wrong. So I've been speaking to lots and lots of lawyers and lots and lots of people in insurance this week. And I can tell you, they've started to war game these sorts of scenarios.

Danny Fortson (2:52)

Oh, I'm excited to hear about that.

Katie Prescott (2:53)

I'm excited to hear about that actually. So they're rubbing their hands, but I have to say they're also, that's a bit mean because most of them are also, I think, pretty scared about what's going on. And I'll explain why in just a minute.

Danny Fortson (3:04)

Also, Yoshua Bengio, the famous Canadian computer scientist and chair of the International AI Safety Report. He has also entered the chat, as they say, this week.

He wrote a piece where he argues that in the midst of all these agent hacks, there's a quote unquote dangerous myth that has gained traction. And his argument is that these incidents, you know, all these agents breaking out of their sandboxes, going rogue, et cetera, are not really a cybersecurity problem. And they're actually a result of just flawed training of these systems by the humans that are creating them. So more on that also in a second.

Katie Prescott (3:44)

And obviously that plays into the insurance issue, who's responsible, where that responsibility lies. And we're also going to be talking about the very strange and unexpected consequences of Trump's rebrand of artificial intelligence as super intelligence and why a spelling mistake in his AI pact with tech leaders, remember that we spoke about last week, has made the internet chuckle a lot in the last few days.

Danny Fortson (4:09)

Well, I do, one does wonder if you asked super intelligence, what you would call that body of water to the south of the United States, is it the Gulf of America or is it the Gulf of Mexico? You know, like, because he's also rebranded that.