Does AI need a kill switch?

SPEAKER_1 (0:01)

The AI boom has been built on going faster. Now, some of its biggest names want to hit the brakes.

Jack Clark (0:08)

I think if we get this technology right, we are going to change the world in so many positive ways.

Gina Neff (0:14)

No other industry would be allowed anywhere in the world to get away with that kind of poor product safety record.

SPEAKER_1 (0:21)

It's World Business Report from the BBC World Service.

I am Bc at DeBio. Coming up, why calls to slow the pace of AI development are rattling investors, and why President Trump wants the spending to continue. Also, Africa's biggest ever share sale, Aliko Dangote wants millions of Nigerians to own a piece of his oil refinery. And Russia's wartime economy may still be growing, but are businesses and ordinary Russians beginning to feel the strain?

So, it's the subject that just won't go away. I'm talking about artificial intelligence. Well, over the weekend, OpenAI, the company behind JPP, said it wouldn't float on the stock market this year amid growing uncertainty around AI. At the same time, OpenAI's chief executive and Elon Musk backed a call from anthropic boss, Dario Amodei, to slow the pace of AI development. Well, those calls have raised fresh questions about just how quickly the industry should be moving. Shares in several leading AI-related companies fell slightly on Monday as markets weighed what a potential slowdown could mean. But despite Monday's falls, though, US stock markets have performed strongly over the longer term. The tech-heavy NASDAQ is up 12% this year. Well, President Donald Trump is firmly against any slowdown in AI spending, and his comments on Truth Social are voiced here by a Bbc producer.

SPEAKER_4 (1:54)

We already have tremendous criminal and regulatory powers over these companies. There is a sick conspiracy going on against AI in data centers, and the only one that is happy about it is China. Whoever wins AI wins. We are leading China and all the others and will continue to do so.

SPEAKER_1 (2:08)

Well, before we delve into the subject, let's bring in Walter Todd, President and Chief Investment Officer at Greenwood Capital in South Carolina, to talk about market reaction. Walter, given the huge amounts of money being poured into AI, I suppose it's not surprising investors have been a little spooked by all this talk of slowing development. We've also seen Microsoft coming out now with its own AI code of conduct too. So how worried are investors really?

Walter Todd (2:37)

Yeah. Well, thanks for having me on.

Actually, the market held up better than I would have anticipated, given the news over the weekend. Certainly, overnight, the futures were down on the tech-heavy NASDAQ 2% or more, but we ended down just about half a percent. You certainly saw pockets like the semiconductors and some other names that were down, five to 10%, South Korea tech-heavy down about 6%. So certain areas reacted, but I thought overall the market kind of held up better than I would have anticipated.

SPEAKER_1 (3:05)

Well, stay with us, Walter, because an unusual feature of the current concern about artificial intelligence is that the warnings are coming from the very companies who are leaders in the field of AI. The BBC's economics editor, Faisal Islam, has been speaking to Jack Clark, a co-founder and head of the public benefit at Anthropic.

Jack Clark (3:25)

AI systems, for a while, have been useful tools with specific skills, like translation or coding or being able to label what's going on in an image. And then in the last few years, people have successfully put all of these skills together into single systems which have started to become much more intelligent and much more general. These systems now, probably about as smart as you or me, or on a good day smarter. The basic risk is, how well do you fancy your chances at being able to deal with the risks of a system that is 100 times smarter than the smartest person on the planet? I think pretty low. Today, these systems are just hacking into websites that host other AI systems. What would happen if a swarm of agents hacked into a bank or into a hospital? That could be really, really dangerous. So now we've had the warning shot. We need to talk about pacing and we need to get a policy regime in place. Get third parties to come in and embed themselves in these companies, audit what we're doing, check our safety and security. So it's not just us writing our own safety and security playbooks, but there are third parties checking it. Number two, get industry together to agree on common standards, but we can implement to reduce the risk of this, and then get governments to agree that those are sensible standards. And then the third step, which will be the hardest one, is get the world's great powers, including the US and China, to have dialogue about this, because these issues transcend any individual country or set of nations.