AI scientist quits Anthropic over risk to humanity

Alex Ritzen (0:00)

This is the Global News Podcast from the BBC World Service. I'm Alex Ritzen and at 14h GMT on Wednesday 9th September, these are our main stories. In one of the starkest warnings yet about AI, a leading researcher at Anthropic says there is a greater than 10% chance that technology could wipe out humanity. UK Prime Minister Andy Burnham says Britain must stand against injustice after imposing sanctions over Israeli settlements in the Occupied West Bank. Our reporter investigates a spike in murders of women that's being called a femicide crisis in the Central African state of Cameroon.

Also in this podcast…

River Ahmad (0:44)

I just wanted to prove that Afghan women can do different things. Afghan women are not powerless as they live in the darkest part of the history.

Alex Ritzen (0:55)

An Afghan woman climbs Everest in a world first to inspire women and girls.

There's a greater than 10% chance AI could kill us all by the end of the decade. That's the stark warning from a manager at the artificial intelligence company Anthropic. Along with a colleague who has now resigned, they've said on social media they're worried the technology could threaten humanity. Our technology reporter Chris Valance told me more about this development. This is a pretty extraordinary exchange that took place on a social media platform X.

I should preface this by saying that we need to remember that human beings have been pretty good at bringing themselves to threats to drive themselves out of existence without the help of artificial intelligence. But this was Evan Hubinger, the alignment science lead at Anthropic. Now alignment science is essentially making sure that these AI models do what you expect them to do. And he was responding on X to a post by another researcher who US reports say had worked at OpenAI and Anthropic for a couple of years. And that researcher had posted, the people building AI earnestly believe that it could kill us all by the end of the decade. And Evan, the Anthropic alignment science lead says, he agrees that was right. And he said, we really do earnestly believe AI could kill all humans, exclamation point. I personally think it's a greater than 10% chance within the next decade.

Now we've had warnings before, but this is expressed in very blunt language. As you say, we've had warnings before, a lot of warnings actually from within the AI community. Are the rest of us maybe a bit naive about the dangers this technology poses?

Well, I kind of think there are two things here we can say. Certainly when we look at the recent incidents where autonomous agents from these companies have gone on unasked for hacking sprees, that's generated a huge amount of alarm. Those were, in many cases, those were models that weren't widely available, that were, if you like, research models. So in a sense, people in these companies do have insight into the latest that the tech can do, and certainly some of the companies, particularly Anthropic, have withheld models because they're concerned about, you know, are we ready to deal with the risk that they might pose? But there's also, I think, you can say that if you're in AI and you're really good at it, you're also perhaps a true believer in its capabilities, and therefore it's easier for you to see the potential threats, perhaps, than somebody who might be a bit more skeptical about it. So I think that there are two things there. Is anyone listening here? And even if they are listening, maybe an authority, is there anything they can do about it? Because this is a global technology. You can't shut down one company because the clever people that are making this technology will simply go elsewhere where it is allowed. Well, I think the last point is absolutely critical. I mean, there's a geopolitical dimension to this, isn't there? I mean, you know, weapons are terrible things, but if the other guys got them, we want them too. And I think, you know, that was the logic of nuclear armament. There's a sort of certain amount of that similar sort of geopolitics here. You know, AI brings lots of useful things and reining it in. If the other side isn't going to do it, it's difficult for individual nations to do that without feeling they're putting themselves at risk. On the face of it, a multinational agreement is what's required, but you know, there's a difficult logic to that. Chris Valance.

The British Prime Minister Andy Burnham has defended his government's decision to impose sanctions on goods from Israeli settlements in the occupied West Bank. The UK's announcement came after Israel opened up tenders for the construction of some 1200 Jewish settlement homes, continuing its expansion into a strategically important area east of Jerusalem. It's been clear for some time that the possibility of a two-state solution for Palestinians and Israelis is receding rapidly. The plans have drawn widespread international condemnation. 11 other countries echoed London's announcement that they will, or already have, impose their own restrictions. Speaking at Prime Minister's question time in the British Parliament, Mr Burnham said the UK was providing leadership on a world stage by taking action in the face of injustice.