Why AI labs gamble with human extinction

SPEAKER_1 (0:01)

This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required compatibility and availability varies 18 plus.

SPEAKER_2 (0:26)

There is so much NFL on ESPN right now.

Monday Night Football, plus pregame and postgame. NFL Live, NFL Primetime. Everything you need to stay on top of every game, every week.

SPEAKER_3 (0:40)

Unbelievable.

SPEAKER_2 (0:42)

And now NFL Network is on the ESPN app. More football, more coverage, more ways to watch. Football's on all season long on ESPN and streaming on the ESPN app.

Ryan Reynolds (0:56)

Hey, it's Ryan Reynolds here for Mint Mobile. Now, I was looking for fun ways to tell you that Mint's offer of unlimited premium wireless for $15 a month is back. So I thought it would be fun if we made $15 bills. But it turns out that's very illegal.

So there goes my big idea for the commercial. Give it a try at mintmobile.com/switch.

SPEAKER_5 (1:17)

Up from payment of $45 for three months, $90 for six months or $180 for 12 month plan required, $15 per month equivalent, taxes and fees extra, initial plan term only, greater than 50 gigabytes, be slow when network is busy, see terms.

SPEAKER_6 (1:26)

Jacob Coxon, our researcher who spent three years working on pre-training at OpenAI and Anthropic, just quit Anthropic, and he stated that both companies are racing straight to self-improving super intelligence and basically gambling with our lives.

SPEAKER_7 (1:41)

Yeah, he bypassed the usual polite resignation letter entirely, you usually see those carefully worded statements about pursuing new opportunities or like spending time with family.

SPEAKER_6 (1:52)

Right, he just walked out of the lab and plainly detailed that the organizations constructing these models are fully aware they are building systems that will eventually outsmart human beings. They're actively choosing to accelerate the process rather than implement safety measures.

SPEAKER_7 (2:06)

Which is wild because, I mean, the focus of his warning really centers on the pre-training phase.

SPEAKER_6 (2:11)

Yeah, the pre-training phase, which is that foundational period where a model just ingests vast amounts of data to build its core reasoning capabilities.

SPEAKER_7 (2:19)

Right, and this is long before any behavioral filters or safety guard rails are even applied to the system.

SPEAKER_6 (2:24)

Exactly. And the bluntness of that exit completely abandons the corporate diplomacy you usually expect from a major tech lab.

SPEAKER_7 (2:31)

It does.

And Coxon added a detail that really changes the whole complexion of the situation. He claims the people actively building these systems earnestly believe the technology could kill everyone.

SPEAKER_6 (2:43)

Yeah, he's not talking about outside critics or politicians.

SPEAKER_7 (2:46)

No, or science fiction writers. He is talking about the specific engineers writing the code, the people optimizing the neural networks, and scaling the compute clusters.

SPEAKER_6 (2:55)

They look at the architecture they are designing, right? They look at the scaling laws that predict how much smarter the model gets with every additional billion parameters.

SPEAKER_7 (3:04)

And they conclude that the logical endpoint is human extinction. So if the people building the system are hitting the panic button and openly discussing their own demise, why is the assembly line speeding up?

SPEAKER_6 (3:16)

If you're not subscribed yet, take a second and hit follow on whatever podcast app you're using. It helps us keep making this. We appreciate you being here. So to answer that, we have to look at what Coxon detailed about what that assembly line is actually producing right now.

SPEAKER_7 (3:29)

The specific capabilities.

SPEAKER_6 (3:31)

Right. He pointed out that these systems will soon be capable of hacking critical infrastructure, acquiring real-world power, and just revolutionizing entire fields overnight.

SPEAKER_7 (3:42)

Okay. But when he says hacking critical infrastructure, we're talking about a very mechanical scenario.

SPEAKER_6 (3:46)

Yeah. A scenario where the models find zero-day vulnerabilities in industrial control systems faster than any human security team could patch them.

SPEAKER_7 (3:55)

Like the programmable logic controllers. Yeah. The PLCs that manage municipal water treatment plants.

SPEAKER_6 (4:01)

Or regional electrical grids. And those are often running on really legacy code. They were never designed to defend against an autonomous agent that can read millions of lines of assembly code in seconds.

SPEAKER_7 (4:12)

Identify a memory leak and just write a custom exploit payload on the spot.

SPEAKER_6 (4:16)

Exactly. And he's directly warning that technical progress is continuing at full speed toward this specific capability without any shared guard rails in place.