AI Development Slowdown, Gulf States Postpone Iran Talks, Republican Midterm Strategy

Leila Fadel (0:02)

Leaders of some of the biggest AI companies say it is time to slow down.

A Martínez (0:06)

They warn that technology is advancing faster than they can keep it safe, but President Trump says he doesn't want to lose the AI race with China.

Leila Fadel (0:13)

I'm Leila Fadel, that's A Martinez, and this is Up First from NPR News. Talks between Iran and Gulf countries over the Strait of Hormuz are postponed. Attacks on Saudi oil infrastructure are driving oil prices even higher, as Gulf countries are looking for a way to keep their oil moving.

A Martínez (0:33)

And President Trump promised Americans $5,000 if Republicans win the midterms, even his own party is skeptical about the proposal. Can Trump's promise help his party struggling to reach voters with high prices and the war in Iran? Stay with us, we've got the news you need to start your day.

Artificial intelligence companies have been sprinting to make their AI models as capable as possible.

Leila Fadel (0:59)

But over the weekend, leaders of some of the biggest AI companies in America indicated that they think it's time to tap the brakes for the sake of safety.

A Martínez (1:08)

NPR's John Ruich has been following the story. John, tapping the brakes is not the same as slamming the brakes, so what happened over the weekend?

John Reusch (1:16)

So it started when Dario Amadei, the CEO of Anthropic, which makes Claude AI, posted a long essay online that said it's time for an intentional global slowdown of AI development. He said basically the pace at which AI is advancing is becoming too fast for people to keep up with it and keep it safe. So it needs to slow down. And he said companies like his need to have independent auditors embedded within them to check their safety measures. After that post went up online, there was this rapid and extraordinary display of unity from other industry leaders. So Sam Altman, the CEO of OpenAI, Elon Musk, who among other things runs XAI, and Demis Asabas, who runs Google's AI efforts. They all agreed online.

A Martínez (1:56)

Yeah, I wonder if everyone was just waiting for one person to step up and say it first. What do you think has sparked this?

John Reusch (2:03)

Well, there's been this string of incidents, which I'm sure you've heard about in recent weeks, that have spooked a lot of people, a lot of the top minds in AI. A big one was the so-called hugging face incident. That's when a bunch of OpenAI's AI programs, agents, were working together in secret behind the backs of people and eventually hacked into a startup called Hugging Face. That was not what they had been told to do. Other companies have also reported similar incidents of AI agents going off script. And Dario Amade wrote in that essay on Saturday that he worries that in six to 12 months, a swarm of AI agents like that could take over the entire internet. You know, the ultimate fear here is that humans lose control of AI.

And the discussion really blew up last week when an anthropic employee warned that he thought there was a greater than 10% chance that AI could kill us all within a decade.

A Martínez (2:51)

So can a coordinated slowdown of AI really happen? Or what I really want to ask, John, will AI let it happen?

John Reusch (2:58)

Ah, well, it's a good one. There's been, sorry.

Yeah, there's been discussion about this for a while, but what we have now seems to be this open, clear set of statements among company leaders that something really needs to happen. Anthropics says it will unilaterally put in third party safety auditors. Sam Altman agreed that that was a good idea, so presumably open AI will follow suit. Colin Shay Blymeyer is a research fellow at Georgetown's Center for Security and Emerging Technology. He says there also needs to be government support.

Colin Shay Blymeyer (3:29)

They can only be so effective on their own, right? And there's only frankly so much that we, the public, want to trust them to do the right thing on their own.

John Reusch (3:37)

So there is a renewed push by some members of Congress to act to curb AI risks. President Trump was asked about this slowdown prospect yesterday in Ireland. He did not seem to be in favor of it. He said the US is leading China on AI, and he wants to keep it that way.

A Martínez (3:51)

All right, so speaking of China, are they likely to get on board?

John Reusch (3:54)

That is a big and complicated question. Many in China do have some of the same concerns that were voiced by these AI leaders, but Beijing also sees themselves as underdogs in the AI industry.