Why The AI Doomers Might Be Right - Robert Wright - #1122

**Chris Williamson** (0:00)
I've told you this before, but you wrote probably the most influential book in my life, which was The Moral Animal, and it was the thing that got me started on the trajectory of thinking about evolutionary psychology, of studying human nature more deeply. Why are you now writing about AI, given your heritage?

**Robert Wright** (0:18)
Well, in some ways, it's an extension of evolutionary thinking, in a couple of senses that I think are so under appreciated. AI is a product of evolution and is still evolving. But the other connection to the moral animal, I think, is first of all, well, the moral animal was about the human mind, and AI does a lot of things that traditionally only human minds have done. The other thing I tried to do in The Moral Animal is highlight what you might call moral biases, kind of self-serving moral biases, the way we all think we're right and the other guy's wrong.
I think if we're going to get through the AI revolution in good shape, among the things we're going to have to do is grapple with that, with what you might call the psychology of tribalism, a little more successfully than we have. So I pay a certain amount of attention to that in this book as well.

**Chris Williamson** (1:13)
What's the central question that you're wrestling with here?

**Robert Wright** (1:19)
Is it true that this technology, which obviously holds the potential to bring great wonders, is also in some respects terrifying and could go badly awry if we don't approach it wisely? I think the answer is yes.

**Chris Williamson** (1:38)
That's exactly why this debate is interesting, that we have this endlessly unresolved potential future. I don't know whether you've seen the graph that I think it's the FT put together, and it was three potential futures from an AI perspective. One results in everything getting blown up, one results in exponential growth, the kind of which we've never seen before, and the other results in a 0.2 percent increase in GDP year on year.
So it's like either very little changes or everything changes in one of two directions.

**Robert Wright** (2:10)
Right. Well, I think it definitely has a potential to massively increase GDP. I also think it has a potential to so destabilize the world, if not do something worse to it, that that just doesn't materialize.
In terms of doom scenarios, I'm agnostic about the sci-fi doom scenarios, but I take them more seriously now than I did before I went into this research project. AI actually taking over and maybe deciding it has no use for us or something. I found much to my dismay that it was harder to dismiss those arguments than I thought. But the thing I'm more confident of is it's just going to be an earthquake. It's going to be destabilizing along a number of dimensions, and that's why we need to approach it with care.

**Chris Williamson** (2:57)
Would you have classed yourself as a sort of AI hopeful going into writing this? What was your predisposition before you got started?

**Robert Wright** (3:05)
I wouldn't say I'm a wildly optimistic person by nature. I tend to focus on potential downsides of things. But again, I had not bought the doom scenarios. You know, I had the Doomer in Chief, Eliezer Yudkowski on my podcast 15 years ago. And at that point, it's interesting, he was in mid-transition. He was moving from Singularity Optimist to Doomer.
I was still saying things like, look, AI, you know, it's not a generic property of intelligence that it has a will to power. We have one because of our unique evolutionary history. I was still asking questions like that. Eliezer was saying they're good questions, but X, Y, Z. I wasn't really persuaded by anything he said, but now I am more respectful of the sci-fi doom argument. So, but in answer to your question, I would have to admit that I go into situations looking for things to worry about. I do. That's my nature.
I think society needs those people, and it needs the other kinds of people, and we need to talk things over.

**Chris Williamson** (4:11)
Jeffrey Hinton fears AI and Yann LeCun doesn't. Who do you think is getting the future more right at the moment?

**Robert Wright** (4:18)
Yeah. I start my book with a conversation I had with Jeffrey Hinton in 1983 Not to betray my age, but the truth is I wrote a piece about AI in 1983 I haven't been paying attention to it ever since, but at that point, Jeffrey Hinton did not have a hint of doom in his voice. In fact, I remember the reason I talked to him, I was talking to somebody, I forget who it was, but they said, if you want to hear the gospel about neural networks, you should talk to Jeff Hinton. I talked to him, he was an enthusiast. He said, I know we don't have much to show right now, but just wait until the microprocessors get really cheap, and we have what he was calling massive parallelism.