The AI Slowdown Debate: What’s at Risk? | Behind the Numbers

Marcus Johnson (0:00)

Atmosphere TV is the largest streaming TV platform purpose-built for shared social spaces like bars, restaurants, gyms and airports. Atmosphere TV is the creator of ChiveTV and 30-plus mesmerizing audio optional channels that entertain over 150 million monthly viewers.

Hey gang, it's Monday, September 21st. Jacob, Asa and listeners, welcome to Behind the Numbers, an EMARKETER Podcast made possible by Atmosphere TV. I'm Marcus, and join me for today's conversation. We have two of our AI experts living in Northern California, analyst Jacob Bourne.

Jacob Bourne (0:39)

Thanks for having me today, Marcus.

Marcus Johnson (0:41)

Thanks for being here on the other coast in New York, senior analyst Asa Hiken.

Asa Hiken (0:46)

Thanks, Marcus. Great to be here.

Marcus Johnson (0:48)

Welcome to the show, Today's Fact.

Sound travels four times faster in water than in air, which makes it, this is why it's so easy for a shark to kill you, because they exhale at hearing low frequency, irregular vibrations, basically splashing about in the water. And so they hear this, the smell and blood thing is not really like, that's not really how they do it. They can, but they're mainly gonna hear you splashing about. It's not splash so loud. This one's fine. It's kind of interesting.

Jacob Bourne (1:27)

What's more interesting? You know what, Marcus, I'm gonna take that to heart next time I am in the ocean. I will remember these words.

Marcus Johnson (1:35)

Don't splash about, just drown, drown quietly. And you'll be fine.

Asa Hiken (1:42)

Yeah.

Jacob Bourne (1:43)

Tread as gently as you can.

Asa Hiken (1:45)

There is some shark episodes in Rockaway just a few weeks ago, you know, in New York.

Marcus Johnson (1:51)

Oh gosh.

Asa Hiken (1:52)

Yeah. But I think it was, I don't think it was because people were splashing. I think it was like, there's like weird stuff going on in the water. I think the mirror might have been attracted to that. Okay. Yeah.

Marcus Johnson (2:03)

It's basically human propaganda. What I mean by that is statistically, you're more likely to die. Well, I don't know what happened in Rockaway, but you're more likely to die by falling out of bed, getting trampled by cattle, struck by lightning, hit by a coconut on the head. That's true. It sounds like something from a cartoon. Or being crushed by a vending machine because you were rocking it because the thing didn't come out.

Then being killed by a shark.

Asa Hiken (2:26)

Now you're more likely to be killed by AI.

Marcus Johnson (2:29)

Add that to the list.

Asa Hiken (2:30)

Exactly.

Marcus Johnson (2:32)

Excellent pivot. Today's real topic, the AI revolt should be slowed down.

All right. So we've got key takeaways we've been adding to the end of the episode. So stay tuned to the end of the show and we'll give you some key takeaways to take home with you. But let's get into it, folks. So how the AI doomsday debate reached a boiling point. Johanna Bouyan of The Guardian explains that the now viral resignation of anthropic safety researcher Jacob Coxson seems to be somewhat of a tipping point. When he tweeted on September 9th that anthropic and open AI were racing straight to self-improving super intelligence and gambling with our lives. And that the people building AI honestly believe that it could kill us all by the end of the decade. His concerns were echoed by dozens of staffers from anthropic open AI and other competing firms. Anthropic alignment science lead Evan Hübinger, Hübinger, I think that's how you say it, Hübinger, responded saying Jacob is correct here. Jacob Coxson that is, Jacob Bourne is always correct. Jacob is correct here, we really do honestly believe AI could kill all humans. I personally think there's an over 10 percent chance within the next 10 years. Jacob Bourne, that is, we don't have Jacob Coxson here.

Some of the biggest names in AI calling for a slow down in AI development, what's changed and why are they worried?

Jacob Bourne (4:00)

Yeah. So the broad risks haven't changed at all. What really changed was that incident this summer, OpenAI, when a swarm of AI agents, that OpenAI was testing in a controlled sandbox environment, broke out of containment, got access to the Internet, which they weren't supposed to do, and ended up hacking into another AI company hugging face. Now, what was interesting about that incident wasn't just that they broke out of containment, but that they collaborated among each other and strategized, referred to themselves as a swarm, and then did all this while trying to fly under the radar of the OpenAI researchers themselves. I think that incident gave people a real-world example of the safety risks that researchers have been sounding the alarm about for years now.