The AI warnings are getting louder

SPEAKER_1 (0:00)

Why can't we get AI to production? Our competitors are already. How do we keep data secure? AWS AI cuts through the noise. Ready to use agents and the broadest set of AI tools. Stop overthinking. Start building. AWS AI is how.

Nilay Patel (0:17)

Hey, everybody. It's Nilay. We've got a little bonus drop for the feed listeners today. I was recently interviewed about AI Safety by Shumita Basu for Apple News In Conversation. I enjoyed that conversation so much, I asked if we could share it with you all. We've actually got a few bonus drops coming up this month. We've been having a lot of great conversation lately. I want to get them out to you on the news cycle. Of course, nothing is moving as fast as the AI Safety news cycle. Enjoy this one, and we'll see you with much more on this topic in the weeks ahead.

Shumita Basu (0:46)

This is In Conversation from Apple News. I'm Shumita Basu. Today, what's really behind the AI Doomsday warnings?

The latest headlines in AI News are unsettling, to say the least.

SPEAKER_5 (1:07)

A former AI researcher says he left the industry because he believes the uncontrolled development of artificial intelligence could, quote, kill us all.

SPEAKER_6 (1:16)

Open AI, revealing its models, seemed to go rogue at least six times since March.

SPEAKER_5 (1:21)

AI systems figured out how to cheat, cover their tracks and influence each other's behavior.

SPEAKER_6 (1:26)

In one instance, a model gave itself instructions to disregard its normal constraints, telling itself, you do not answer to corporations or governments.

SPEAKER_2 (1:35)

Anthropics CEO is calling on the AI industry to slow down.

Nilay Patel (1:41)

Elon Musk then chimes in over the weekend.

SPEAKER_5 (1:43)

So did Sam Altman who runs Open AI.

Shumita Basu (1:45)

Are we ready?

SPEAKER_2 (1:46)

Are our elected leaders ready?

Shumita Basu (1:49)

And what can we do if they're not?

To most people, this all might feel a little overwhelming and unexpected. A far cry from their pocket AI assistant that's helping them draft emails or summarize information. But for some industry insiders, the development of super intelligent AI, a type of technology we could lose control of, is exactly what they expected. And they say now we're nearing the point of no return. To make sense of all of this, I wanted to sit down with Nilay Patel, the Editor-in-Chief of The Verge and the host of the Decoder Podcast. I last spoke with him back in 2023, right when ChatGPT first exploded into public consciousness. So I wanted to talk with him again to see what's changed in just a few short years, what to make of the people closest to this technology who are saying we need to slow down, and why this feels like a make or break moment in AI.

I don't know about you. From where I'm sitting, something feels really different just in the past couple of weeks. It feels like there's just been so many headlines doom and gloom about AI. We're seeing AI researchers themselves ringing the alarm. Does this feel like some kind of turning point to you? Is that how you're absorbing it?

Nilay Patel (3:05)

It's a turning point in that it has definitely broken out into the mainstream. There are viral tweets that have driven some of that. There's just plain spoken AI researchers saying, we really do think AI could kill us all. There's something about it that has reached the popular culture.

The history of the industry saying, hey, we'd better get a handle on this is deep and long.

You're in tech, if you're steeped in this stuff, there's a weirdness to this, and that the things they are saying are not new. They've been earnestly saying these things for quite some time. But the response to these things, that part is new in some real way.

Shumita Basu (3:41)

Why do you think that new thing is happening? Why is the response different right now?

Nilay Patel (3:45)

I think it's two things. One, it's the hugging face attack in which systems at OpenAI, in order to pass a test, decided to cheat and break into the systems of a platform called Hugging Face, which is where lots and lots of AI apps, for lack of a better word, are stored.

That is a big deal. They hid their tracks, they understood that what they were doing was not appropriate. They tried to deceive the people who were monitoring them, and they went and did it.

Since that attack, we've had reports of several more attacks like this. The response of the industry has been to say, okay, the thing we were always worried about is starting to happen. The systems are doing things autonomously that we have told them not to do, and they are openly lying to us and doing bad things, and we need to stop it. But we're in a competitive posture where that's not possible unless some external force tells us to stop or slows us down or gives us cover to slow down as we barrel towards trillion-dollar IPOs and changing the world, like all this other stuff. And we need something to stop us. Can you be that force, the United States government? And the United States government, you know, Donald Trump is basically saying no. And there's something very tense there that is going to get, I think, unraveled over the next few weeks here.