#215 - Nick Bostrom - "We Are Creating Superintelligence": The Existential Risk is Real

Nick Bostrom (0:00)

We will have extremely powerful AI, superintelligence, machine minds that leave our biological intelligence in the dust. And so then you wonder what that powerful AI could do, and probably it could do whatever it wants to do.

That could reduce in the elimination of humans, either just as a side effect, or they might more actively eliminate us if they thought we might possibly pose a threat.

Peter McCormack (0:28)

But does it ever detach itself from our goals to become its own goals? Does it become conscious in itself?

Nick Bostrom (0:33)

Whether it will have its own goals, I mean, that depends on whether we solve this alignment problem, whether we succeed in embedding our values into the AIs that we're building. And that's an open question.

Peter McCormack (0:50)

What are the scenarios that would be most concerning?

What are the scenarios that you think would lead to the existential risk of our civilization?

This show is brought to you by my lead sponsor, Iron, the AI cloud for the next big thing. Iron builds and operates next generation data centers and delivers cutting edge GPU infrastructure, all powered by renewable energy. Now, if you need access to scalable GPU clusters or are simply curious about who is powering the future of AI, check out iron.com to learn more, which is iren.com. Right. Well, listen, thank you for doing this. I've wanted to meet you and talk to you for a long time. When you make a podcast, there's certain people you're like, I've got to talk to that person. So thank you for this. And obviously, very interesting times. So I really want to start, which is probably something you thought a lot about, you think a lot about is AI. It's the subject of the moment.

Nick Bostrom (1:50)

It is.

Peter McCormack (1:51)

It's touching all our lives. And we seem to be at a moment where there's a camp that's freaking out about it, and there's a camp that's very excited about it, and there's some people in the middle of it recognizing both worlds.

I just want to start on the camp who's freaking out about it. What do you think? You're somebody who thinks a lot about life and humanity and civilization.

Nick Bostrom (2:12)

Well, a lot of my friends and colleagues are in that camp. They're freaking out camp.

We can see the rapid pace at which AI is advancing. And it doesn't take a huge leap of imagination to think that if it keeps going at that pace, even just for a couple more years, then we will have extremely powerful AI, superintelligence, machine minds that leave our biological intelligence in the dust. And so then you wonder what that powerful AI could do. And probably it could do whatever it wants to do. That then raises the alignment problem, right? Like we need to make sure that we design these minds so that what they want to do is something that is also good for us and that helps us get what we want. And that looks like a difficult problem. So then, yeah, they freak out.

I'm, as you, I guess, suggested in this middle ground where I'm simultaneously both very excited about the upside of this technology and also worried about how things could go wrong if we don't manage the transition well.

Peter McCormack (3:27)

Do you think some of the worry about the alignment issue is because you could say that as humans, we have an alignment problem already?

Nick Bostrom (3:37)

We do. I don't know whether that's the reason for the freakout. I think in some ways it's the other way around that some of the freakout might be partially derived from a failure to appreciate human misalignment. So if you imagine sending some human off to become superintelligent and be in a position to decide the future unilaterally, that also would seem pretty scary.

A, because they might, if you pick the random human, not all humans are nice. Even if you pick a good human, who knows what would happen to them if their brain developed to super, maybe their personality would change or they would arrive at new conclusions that could also be scary. So if we consider two possible paths into the future, one which involves the development of machine superintelligence, which might fail to align and that could be dangerous. But I think some of those risks, not all, but it could also arise if you imagine humans running the show. And then you have the additional complication, of course, that there is not just one human entity that is staring this, but there are many different ones with competing goals that might then fail to coordinate. They might actually use the powerful technology at cross purposes to wage war, to oppress.