AI Could Kill Us All? The People Building It Are Warning Us

Michael Smerconish (0:00)

Energy, it's more than a utility. It's what powers all life's moments and possibilities for generations to come. It's what helps build brighter futures for all. So no matter how the world and its energy needs change, Southern Company's commitment to meeting those needs stays the same. They're continuously looking forward, forecasting how to support communities and deliver clean, safe, reliable and affordable energy. And that means together, we're ready for what's next. Learn more at southerncompany.com.

Southern Company, building the future of energy.

SPEAKER_2 (0:33)

Hey, before your Q3 call, I've got the campaign brief ready, built from last quarter's data and the competitive landscape.

SPEAKER_3 (0:40)

Great. Did you include the differentiation angle the CMO asked for?

SPEAKER_2 (0:44)

Already in there. Three angles no competitors using right now. Just need your approval.

SPEAKER_3 (0:49)

Approved. Thanks. Oh, agents, where would we be without you?

SPEAKER_2 (0:54)

Hmm, somewhere with a lot more tabs open.

SPEAKER_3 (0:57)

Create your first Monday agent in minutes at monday.com. Welcome to The Smerconish Podcast for Independent Minds.

Michael Smerconish (1:08)

So a moment ago, I was listening to the news feed here on SiriusXM POTUS. And the lead story, you heard it perhaps, was about artificial intelligence. In our studio, we've got an enormous 98-inch flat screen television. We rely on Apple TV, which breaks it into quads. And we show CNN, MS Now, Fox News and the BBC. And at the top of the hour, all four images, all about artificial intelligence. And understandably so. I intend to talk about it right now. It's the lead story. It has been for a couple of days. But I have to say, everyone is missing the mark.

And I'm included among those who missed the mark. I talked about artificial intelligence on CNN, led with it as a matter of fact. Our friend Richard Clark was my lead guest, and he was terrific for reasons I'll explain. It was one day after the 25th anniversary of September 11 that I found myself conversing with Richard Clark. Richard Clark, who famously, in a 9-11 commission hearing, a few years after September 11, said, look, we failed the country. I failed the country. Because the alarm bells, I'm paraphrasing, but the alarm bells were all ringing.

And we didn't do what was necessary. And when I asked him, do you feel the same about this? He said, absolutely.

But what people are missing and what I missed in the commentary that I delivered on Saturday, and frankly, I was just consuming too much. Nobody is reading the report. No one is reading the report. I'm going to share with you the report that I'm making reference to. Let me set the stage. You're sophisticated. You're POTUS listeners. I'll bet you know most of this. But for the benefit of those who were tuned out and paying attention to the NFL opening weekend, it begins with Jacob Coxon, the 27-year-old, the Cambridge Mathematics who spent three years at OpenAI, then Anthropic, who quit last Tuesday saying that the people in the building, the building at Anthropic, think Claude, earnestly believe it could kill us all by the end of the decade. And then the part, because we've heard that before, the part that really, I think, brought it home is that a researcher who is still at Anthropic, Evan Hubbinger, who runs alignment stress testing at Anthropic said publicly the very next day, Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think that it's greater than 10% in the next decade. And then over the weekend, Dario Amadei, the CEO of Anthropic, the man with billions of reasons to say the opposite, sat down with Anderson Cooper and essentially said like, yeah, both of them are right. And he went so far as to say, I agree with Jacob more than I disagree with him.

SPEAKER_4 (4:19)

Obviously, the reason we're talking is these headlines that have come out.

Jacob Coxon, who recently resigned from Anthropic, made headlines last week saying, the people building AI earnestly believe that it could kill us all by the end of the decade. There's a current employee, Evan Hubbinger, an alignment science lead. He agreed, saying, we really do earnestly believe AI could kill all humans. I personally think it is greater than 10% chance within the next decade. Do you believe, earnestly believe, that AI could kill all humans? And what percent chance do you think it is?

Dario Amodei (4:52)

So, you know, it's funny. I agree with Jacob much more than I disagree with him. You know, and it's an interesting resignation because when he left, he said, you know, I think Anthropic is the most responsible player, right? I think Anthropic is the most aware of these issues. He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast. Now, you know, I think I already said, you know, these sort of unconditional probabilities, you know, I used to kind of talk in that way because it's kind of an easy way to express, I think there's some chance that things go wrong, but, you know, but, you know, but, but, but it's not, it's not all that high, but again, I think it's more illuminating to think in terms of, you know, how do you decompose those probabilities? What are the paths in which things go well? And what are the paths in which things go poorly? And so instead of saying, you know, it's a 10% chance, that sounds like it's a roll of the dice, I think it's much more illuminating to say, well, that forks into different paths. And if we take the right paths, then the chance of something going wrong is very low.