Is there a 10% chance AI “could kill all humans”?

Katie Prescott (0:02)

Danny Fortson.

Danny Fortson (0:03)

Yes.

Katie Prescott (0:04)

You don't look like you're in your normal spot in Silicon Valley with your lovely plant behind you.

Danny Fortson (0:11)

I'm not.

Katie Prescott (0:12)

You look, if I may say so, like you're in a hostage video.

Danny Fortson (0:15)

Don't worry. I don't have a newspaper with like today's date on it to show the proof of life or whatever. I meant, it's actually a Ramada. Ramada Inn. What do you guys call it?

A Ramada.

Katie Prescott (0:29)

A hotel. You're in a hotel.

Danny Fortson (0:31)

Yeah. In scenic Calgary. I'm here for a story covering another tech billionaire. So I am safe. I appreciate your concern.

Katie Prescott (0:39)

Well, let's get to the pod.

Danny Fortson (0:40)

We shall. Let's get to it. Hello and welcome to Times Tech, where we unpack how technology is reshaping our everyday lives. I am Danny Fortson.

Katie Prescott (0:50)

And I'm Katie Prescott. And the cheery news we bring you this week is that there is more than a 10% chance that AI could kill humans.

Danny Fortson (1:00)

Cool.

Katie Prescott (1:01)

Cool. And that's according to a key figure at Anthropic, the chap who leads research on what's called alignment, basically making sure that the models do what they're told to do. So that might win the prize for the most extreme warning about AI of the week, but he is not alone. And the last few days have been peppered with news of a potentially very dangerous new era of AI.

Danny Fortson (1:25)

Yeah, this whole conversation started when OpenAI released a powerful new model called Astra, which Jensen Huang of NVIDIA hailed as AGI, artificial general intelligence, you know, human level intelligence. We're here, we've done it. We'll be getting into the details of all of that, including why it's got tech bosses sounding the alarm yet again.

Katie Prescott (1:47)

And the question is really how seriously should we take this? But before we get into that, we wanted to tell you about our next episode of The Interview, which is the part of the tech podcast where we speak to some of the biggest names in tech shaping all of our futures. And it doesn't get much bigger than this. I've been speaking to...

Danny Fortson (2:05)

That's my big name, Klaxon.

Katie Prescott (2:07)

That is the big introduction that we need to Bill Gates, the Microsoft founder. And we had a fascinating discussion about why he thinks AI is completely different to any previous tech, why it poses a threat to humanity, and who should be responsible for controlling it. And that episode is going to be out next Tuesday from 6 a.m. London time. So make sure to listen or watch on our YouTube channel.

Danny Fortson (2:33)

Yes, indeed. So all very exciting. But let's get back to today's episode because we're also going to be talking about a creepy new documentary trailer that's been released, which has everyone in Silicon Valley talking. It's about Elizabeth Holmes. Remember her, the disgraced founder of Theranos, which is this health tech company that was once worth nine billion before it all went downhill and it was exposed as a fraud. It's a pretty crazy story and feels relevant to this AI moment that we're in.

Katie Prescott (3:03)

Should we start with the light news though?

Danny Fortson (3:05)

Yeah, let's do that.

Katie Prescott (3:07)

So when I woke up this morning and jumped onto X, as you do, and started scrolling through some of the top posts.

I was greeted by one that kept coming out time and time again, some people retweeting it from a guy called Jacob Coxson, who said he was resigning from Anthropic because he'd spent the last three years there doing pre-training research, also OpenAI by the way. And he said, neither company is acting responsibly. They're racing straight to self-improving super intelligence and gambling with our lives. Now, of all of the retweets from Jacob of this message, which, you know, and there have been a number of people, haven't there, over the past few years who've resigned from the Frontier Labs.

Danny Fortson (3:54)

Yeah, this has become kind of like an oeuvre. Like it's a pretty large body of these kind of like, I'm quitting, here's why, we're all up a very famous creek and I'm going to go, you know, write poetry in the mountains and wait for the end times.

Katie Prescott (4:10)

So, relatively well-throdden path. But what stopped me in my tracks was a retweet from a guy called Evan Hubinger at Anthropic, who leads what's called alignment. So, this is making sure that the AI is literally aligned with what whoever's created it wants it to do. When it's not, they call it misalignment. Evan says, Jacob is correct here. We really do earnestly believe AI could kill all humans.