Is AI a threat to humanity?

Vishala Sripathma (0:01)

Is AI a threat to humanity?

Susannah Streeter (0:03)

Why would anyone work there if they genuinely think they could be responsible for killing off the human race?

Chris Miller (0:08)

A lot of the discussions of regulating AI happen at a 30,000 foot level that's separated from the actual challenges.

Vishala Sripathma (0:15)

It's World Business Report from the BBC World Service.

I'm Vishala Sripathma. On the way, a top researcher at Anthropic says AI poses a threat to humanity, so how worried should we be? Plus a blast from the past, Apple brings back the foldable phone and the Indian food inspector has become an unlikely celebrity.

So, does artificial superintelligence really pose a risk greater than nuclear weapons? Is there a significant chance of a Chernobyl-sized catastrophe? Many experts in AI say the fears are overblown, but on Wednesday, a top safety researcher at Anthropic warned that the technology is advancing so quickly that there is a greater than 10% chance it could kill all humans within the next decade. Evan Hubinger says he is concerned that AI systems could eventually become capable of improving themselves faster than humans can understand or control them. His comments were prompted by a post from Jacob Coxon, a researcher who recently left Anthropic after previously working at a rival firm, OpenAI. In a lengthy social media thread, he explained why he left compelled to walk away. His tweet has been voiced by our producer Neil.

Jacob Coxon (1:38)

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible. But I hear the same people express fears privately.

No other human activity poses this level of danger.

Vishala Sripathma (2:00)

Dame Wendy Hall, who is a member of the UN's advisory body on AI, says she is shocked by the comments.

Susannah Streeter (2:08)

Who is going to rein these companies in? Some of it could just be PR or marketing, right? You never know with them, you can't trust them. But how a human being can make a statement about our company has a 10 percent chance of killing off the human race. Why would you want to say that? Why would anyone work there if they genuinely think they could be responsible for killing off the human race?

Vishala Sripathma (2:28)

It's certainly a topic that's got lots of us talking. We spoke to people in the north of England and asked them whether they were concerned about the increasing abilities of AI models.

SPEAKER_5 (2:39)

Some jobs like they are starting using AI in a state of employer or some people. So it makes people lose their jobs. And about the chargeability, I use it most of the time, but I'm not depend on it like 100 person.

SPEAKER_6 (2:54)

It's a slippery slope because it depends about who essentially programs it and the people that are programming it right now don't really fill me with confidence. AI companies are like open AI. Essentially, they're just mass data farms that are just building profiles on people. And if you tie that into a citywide system, then you are building a mass surveillance system on the population.

SPEAKER_7 (3:17)

I'm a little bit worried about AI. I think already there's been plenty of examples of AI doing things that we didn't expect it to or AI sort of breaching boundaries that we kind of don't want it to yet. I think that it's probably not as scary as we think, but I think there's definitely the prospect of it being pretty scary.

SPEAKER_8 (3:35)

I absolutely love it, yeah. I think it's great for supporting me teaching, great now that they're finding things for cancer.

Vishala Sripathma (3:42)

So a mixed view, a mixed lot of opinions there. Professor Melanie Mitchell is a computer scientist at the Santa Fe Institute and joins us now. Welcome to the programme.

Melanie Mitchell (3:54)

Thank you for having me.

Vishala Sripathma (3:55)

So you've previously poured cold water over these concerns. How are you feeling now? Because these big brains behind AI are worried. It's rather unnerving.

Melanie Mitchell (4:06)

Well, to be honest, this is nothing new. This kind of thing, these exact same statements, the 10% risk of human extinction and so on, have been being predicted for decades now.

And we haven't certainly haven't seen it happen. We haven't seen that AI systems are getting closer and closer to killing off all of humanity, which would be quite a feat, to be honest. But there are certain risks that are coming to the fore now that I think do need to be addressed.