He Warned AI Could Destroy Us. Now The Industry Is Listening — ft. Nick Bostrom

SPEAKER_1 (0:04)

What's driving the Markets this week?

Ed Elson (0:07)

What's on investors' minds as they look ahead?

SPEAKER_1 (0:10)

Find out on the Markets podcast from Goldman Sachs. A breakdown of market moves and macro signals in 10 minutes or less.

The Markets podcast from Goldman Sachs.

SPEAKER_3 (0:31)

Strength is one of the most important investments in your long-term health. That's why I love Orange Theory Fitness.

Every one-hour Orange Theory workout helps you build muscle, burn fat and take control of your health, guided by a coach every step of the way. No wondering what to do next. No intimidating gym floor. Just strength built for life. Visit orangetheory.com/spotify to learn more and get your first class free. Terms and conditions apply. See website and participating studio for details.

SPEAKER_4 (1:01)

Parenting unlocks some pretty impressive skills, like carrying grocery bags, a soccer ball and a sleepy kid all at once. Good thing the newly designed Chrysler Pacifica has a few tricks of its own. Hands-free power sliding doors and lift gate, rear seat entertainment and stow and go seating and storage that adapts faster than your family's weekend plans. Visit chrysler.com to learn more. Chrysler is a registered trademark of FCA US LLC.

Ed Elson (1:41)

Welcome to Prof G Markets. Last week, a former Anthropic researcher revealed that employees at both OpenAI and Anthropic believe that AI could quote, kill us all by the end of the decade. His post quickly went viral and several other AI researchers came forward to say that they actually shared the same concerns.

Then over the weekend, Anthropic CEO Dario Amadei published an essay calling for the industry to slow down the development of AI models. Sam Altman said he agreed with Amadei and added that Open AI will not be going public this year.

There is now a growing debate over whether these fears are justified or overblown. So we wanted to hear from one of the people who has been studying this problem longer than perhaps anyone. His 2014 book, Superintelligence, helped shape how the world thinks about AI.

It influenced many of today's AI leaders, including Elon Musk, Sam Altman, and Ilya Tsitskeva, who even named his company after the concept. Our guest is one of the most influential philosophers of our time, and he is here to help us make sense of just how dangerous AI could become, and whether the warnings we are hearing today deserve to be taken seriously. This is our conversation with Nick Bostrom, AI philosopher and bestselling author of Superintelligence and Deep Utopia. Nick, thank you so much for coming on the show. It really is an honor to have you, especially at this time, where your research and your writing is so relevant. I guess we should start with the tweet that went viral. Jacob Cox and tweet the now former anthropic researcher who said that the people building AI, quote, earnestly believe that it could kill us all by the end of the decade. He said, this is not a marketing stunt. Then the world seems to sort of blow up, at least the global conversation blows up. Let's just start with your initial reactions to that tweet and how it has impacted the AI conversation.

Nick Bostrom (3:46)

Well, let's see if we can try to make sense of this situation. It is a very confusing and perplexing moment I think for humanity. We being sort of close to the potential birth of superintelligence.

The idea that there could be significant risks associated with this, including existential risks is quite widespread, I think, amongst people close to this technology and in the frontier labs. As I agree that it's not a marketing stunt, I think it's coming from a sincere place. A sense that we are getting in quite deep here, and we should really pay attention to what is happening.

Ed Elson (4:30)

Elon Musk is saying kind of two different things. On the one hand, he tweeted back in 2014 that he read your book, and that he thought that AI is potentially more dangerous than nukes, and then he retweeted it quite recently. He even said that he agreed with Dario Amadei in terms of the size of the problem. But then he also said that he thinks that it might be a marketing stunt too. It's not totally clear where he stands on this. I just want to play you this clip of what he said.

Here's the clip.

SPEAKER_6 (5:04)

It certainly is like some crazy 4D chess to say there's whatever 10% chance of annihilating humanity. But by the way, how much allocation would you like in our IPO?

Ed Elson (5:18)

Do you think there are any merits to that argument?