The A.I. Researcher Whose Rebellion Is Changing Everything

SPEAKER_1 (0:00)

This podcast is supported by the Conservation Fund. For over 40 years, the Conservation Fund has raced to protect at-risk land before it's lost. The forests, farmland, open spaces, and places that wildlife call home. They believe conservation shouldn't be a choice between protecting nature and supporting people.

It should advance both in harmony. With over 9 million acres protected across all 50 states, the Conservation Fund secures irreplaceable lands people love. Learn more at conservationfund.org.

Natalie Kitroeff (0:34)

Can you just read the third post from your thread?

Jacob Coxon (0:39)

Okay. The people building AI honestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.

If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately. No other human activity poses this level of danger.

Natalie Kitroeff (1:00)

From The New York Times, I'm Natalie Kitroeff. This is The Daily.

SPEAKER_4 (1:04)

A dire warning about AI and humanity from a former anthropic researcher who suddenly resigned.

Natalie Kitroeff (1:12)

Over the last week, a young AI researcher quit Anthropic and posted warnings about the risks of artificial intelligence that went viral.

SPEAKER_5 (1:22)

I want to better understand what you mean when you say AI could kill us all.

SPEAKER_6 (1:27)

I mean, how likely is this doomsday scenario that you've presented?

SPEAKER_4 (1:32)

And he's not the only one to sound the alarm.

Natalie Kitroeff (1:35)

And kicked off a crisis that culminated this weekend in a call to action by the most prominent leaders in the industry.

SPEAKER_7 (1:43)

We begin with the stunning news from Anthropic CEO, Dario Amadei, that he is urging a slowdown when it comes to the development of artificial intelligence. It was a pronouncement that his chief competitor, OpenAI CEO Sam Altman and Elon Musk quickly agreed with.

Natalie Kitroeff (2:01)

Those leaders came out in favor of a global slowdown in the development of artificial intelligence.

SPEAKER_8 (2:08)

I agree with Jacob much more than I disagree with him because when he left, he said, you know, I think Anthropic is the most responsible player, right? He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast.

Natalie Kitroeff (2:26)

Today, we talked to the researcher who got us to this milestone moment, Jacob Coxon.

It's Monday, September 14th. Jacob, hi.

Jacob Coxon (2:40)

What's up?

Natalie Kitroeff (2:40)

Can you hear us?

Jacob Coxon (2:41)

Yeah, I can hear you fine.

Natalie Kitroeff (2:42)

Perfect. Great. Thanks for being here. By now, I think you've been on maybe every major news network saying essentially that AI could pose an existential threat to humanity.

We've seen industry leaders grappling with that, lawmakers as well. Did you anticipate this kind of response?

Jacob Coxon (3:02)

No. I just wanted to tweet what I was thinking.

I expected it to maybe go a bit viral among people that already shared this belief, but I did not expect it to hit the whole world. It's kind of surprising to me that it was my tweet that happened to trigger a particular wave of talking about this, because of course, the experts have been talking about this for so long.

Natalie Kitroeff (3:26)

Okay, well, we're going to get to the tweet, but I want to start this conversation by just asking you about your first experience with AI. When was the first time that you heard about this technology, do you remember?

Jacob Coxon (3:42)

I remember the first time I heard about it really working was AlphaGo in 2016

SPEAKER_9 (3:47)

A Google supercomputer has beaten the world's best player at the 3,000-year-old Chinese board game called Go.

Jacob Coxon (3:56)

So that was when DeepMind's Go playing AI beat Lee Sedol, the champion Go player in a match.

SPEAKER_10 (4:02)

The aim of the game is to capture as much territory as possible on a 19 by 19 square grid.

Jacob Coxon (4:08)

And that was pretty crazy.

SPEAKER_10 (4:09)

There are more possible moves on the board than atoms in the universe.

Jacob Coxon (4:13)

I studied math in college, but I sucked at Go. It was like exciting to hear that AI could win at that.

And then GPT-3 released during the pandemic when I was graduating.

SPEAKER_9 (4:26)

Chat GPT, which is a new product from OpenAI.

SPEAKER_1 (4:31)

It's a remarkable beta search.

SPEAKER_11 (4:33)

It can interpret human language, answer questions. It can even generate written texts, essays, books, news articles, and even computer code. And it's really good.

Jacob Coxon (4:45)

By the early 2020s, so by five years ago, it was pretty clear that this was going to be a pretty big deal. This was going to be more important than doing math or basically anything else.