Special Report: Will AI Wipe Out Humanity? (And Who Profits)

Alex Kantrowitz (0:00)

Is AI going to kill us all? We'll cover it extensively with the appropriate context and nuance on a Big Technology Podcast Friday edition coming up right after this.

SPEAKER_2 (0:11)

There is so much NFL on ESPN right now.

Monday Night Football, plus pregame and postgame. NFL Live, NFL Primetime. Everything you need to stay on top of every game, every week.

Ranjan Roy (0:25)

Unbelievable.

SPEAKER_2 (0:26)

And now NFL Network is on the ESPN app. More football, more coverage, more ways to watch. Football's on, all season long on ESPN and streaming on the ESPN app.

Alex Kantrowitz (0:40)

Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional cool headed and nuanced format.

Crazy week of AI news this week. It's hard to fully capture the rolling craziness that's happened in the AI industry, everything from the millennium problem being solved by open AI and the dispute over credit to what eventually overshadowed it, which is an entropic researcher came out and said he's leaving because he believes AI is going to wipe out humanity, or at least there's a 10% chance that it's going to do that. And that eventually set off a international, I don't know, you'd call it a freak out maybe, where everyone from policy makers to senators to academics, thought leaders, journalists and people in the AI industry expressed an opinion. And there's been so much madness about it that we're going to try to do what our show, hopefully, when we do what we're supposed to do does best, which is bring some nuance and some coolheadedness to this crazy conversation. And hopefully at the end of this special report, leave you walking away with some groundedness in it so the extreme opinions aren't the ones that rule the day. We're joined as always on Friday by Ranjan Roy of Margins. Ranjan, you're still here.

Ranjan Roy (1:58)

I'm still here for now. Human extinction has not happened yet. So until it does, we will podcast every Friday.

Alex Kantrowitz (2:09)

As is our duty. As is our duty.

Ranjan Roy (2:11)

As is our duty while humanity continues to exist.

Alex Kantrowitz (2:16)

We will bear this burden once required of us by the Podcast Gods. Let's get right into the story just so that the appropriate context is out there. This is from the Wall Street Journal. Anthropic researcher quits over out of control AI fears. An anthropic researcher is quitting the artificial intelligence industry over fears that the lab and its competitors are racing to build systems they won't be able to control, a sign of mounting safety concern within top AI companies. Jacob Coxon, a researcher who specializes in creating new AI models by having them consume vast amounts of data said Tuesday he's leaving the company because he doesn't want to participate in an industry-wide rush to build AI systems that can improve themselves, worried such systems could spiral out of control and destroy humanity. He followed it up in a tweet, said, I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving super intelligence and gambling with our lives. Don't underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight and acquire real power and resources. We have all witnessed the progress in each of these domains and progress is not slowing.

We're going to get into all the details here, but Ranjan, I mean this thing has gone just like absurdly viral, 165 million views to date and counting.

What was your view of this when you saw it come out? And are you surprised that it has hit such a chord that it's reached so many people in such a short amount of time?

Ranjan Roy (3:56)

I think the escape velocity side of it was a bit surprising, because again, we have been talking about this for a long time. In fact, I spent part of my Labor Day weekend reading the meter research and going through it and trying to understand what kind of real vulnerabilities we actually have around rogue agents escaping containment. And then obviously to see this kind of take off, it is interesting to me. Part of me like the fact that this happened right after Labor Day and everyone's summer vacations are over and they're back fully at work. I do wonder if it has a little to do with it.

Do you think, is that a crazy theory?

Alex Kantrowitz (4:41)

Well, I think the fact that this hugging face story found a way to have life over a month and sort of primed people for a moment like this, you know, might have had something to do with it. Here's a conspiracy theory that I like that I don't think is true, but it could possibly be that, you know, the owner of X probably has something to gain by something like this going so viral, either by forcing OpenAI and Anthropic to slow down or potentially having regulation come in that sort of entrenches those that have built large data centers already and maybe hit a button at X headquarters to make it blow up. But I don't know, that might be a little step too far.