Anthropic: AI could escape human control

**Alex Ritson** (0:00)
This is The Global News Podcast from the BBC World Service. I'm Alex Ritson, and on Friday the 5th of June, these are our main stories. One of the co-founders of the US artificial intelligence company, Anthropic, has warned that AI systems will soon be able to develop without the aid of humans. President Putin's handling of the Russian economy is in the spotlight at the annual Economic Forum in St Petersburg, and Mali is banning the use of motorcycles outside urban areas in a bid to combat Islamist militias.
Also in this podcast...

**Johnny Vorpsey** (0:39)
We are talking for a project that will be built inside one of the most important coastal wetlands, and it's part of the protected area.

**Alex Ritson** (0:48)
Why a massive coastal development project linked to Jared Kushner, the son-in-law of US President Donald Trump, is facing mounting resistance from protesters in Albania.
We begin with a warning about the rapid progress of artificial intelligence from one of the key players in the industry. Anthropic, a major AI developer best known for its clawed assistant, says its latest most advanced systems are beginning to show signs they could escape human control. It says its AI models are dramatically speeding up the rate at which they teach themselves, removing human input from the process. The US company is proposing a slowdown in building them and global rules to contain these systems. One of Anthropic's co-founders is Jack Clark. He's been speaking to the BBC's Faisal Islam.

**Jack Clark** (1:42)
Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal in the car. And what we're saying is we want to do some of the work required to build that brake pedal so we in the world have an option. It's not obvious today that you want to do that, but absolutely, as you say, at some point in the future, you might say, let's get all of the benefits we can for, say, biology and medical research, and let's take a pause or take a moment on AI research where we can absorb the societal changes implied by this.

**Faisal Islam** (2:13)
A number of Silicon Valley AI companies are raising tens of billions of dollars, including yourself, at valuations of a trillion-dollar class. And some argue that these are sort of great narratives that help you sell your kit to the world's biggest companies and walk away with these epic valuations.

**Jack Clark** (2:32)
Well, all I can say is I'm a father, you know, my second child. I came back from paternity leave and my reaction was something really important is happening here and we need to tell the world. This is part of our public benefit mission, is tell the world what we're seeing inside companies with this unusual technology. That's the motivation for this.

**Faisal Islam** (2:51)
Are you worried for your kids if this message doesn't get out?

**Jack Clark** (2:55)
I'm not worried for my kids if this message doesn't get out. I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean. They're going to mean that there are potentially great benefits, there are also risks, they are also going to change things about society and they're going to need to change aspects of policy. We have to have that conversation. I think the story is generally, any profession is now going to be able to work with AI systems and change how work happens as a consequence, which also means any single profession has some potential for AI to come in and augment or automate different types of work. We're trying to measure that. Right now, there aren't clear measures that say that augmentation or automation is happening at large scale, but clearly that's the implication of this, and we're trying to share data about it so that we can see if this shows up in the broader economy.

**Faisal Islam** (3:48)
OK, one last question. I mean, your advice for a confused young person thinking that the jobs that they had set sail to try and get, that they are simply not going to exist, and they're watching instead the sort of share options of Silicon Valley kind of go to the moon and thinking, what am I going to do? Am I even going to have a career? What should they do?

**Jack Clark** (4:08)
Develop a hobby. Anyone who has a hobby has something that they're passionate about and that they know more about than most people. And with that hobby, you can have curiosity, you can have ideas, and you can use that to really get the most out of these AI systems. And I am sure turn that into like amazing jobs, jobs that don't even exist yet. And it requires you to experiment with the systems and have that curiosity.