Roman Yampolskiy vs Joscha Bach: Can Anyone Control Superintelligence?

Joscha Bach (0:00)

All right.

Brian Keating (0:00)

Today, we are joined for what promises to be an interesting conversation between two intellectuals I have great respect for, both past guests on the podcast, and making his second appearance, it's Joscha Bach. Joscha, how are you doing today?

Joscha Bach (0:14)

Hello. Thank you for having me on.

Brian Keating (0:16)

We had a great conversation about the connectome, about the ghost in the machine over the summer and the spring, and that was well received. Got more attention on Reddit. People are very opinionated about you on Reddit over there, as they are about almost anything. Then Roman Yampolskiy needs no introduction, most popular guest ever on many podcasts, and now has his own and has hit 200,000, play a DJ song.

Roman, welcome back to the Into the Impossible Podcast, your third appearance.

Roman Yampolskiy (0:51)

Thanks for setting it up.

Brian Keating (0:53)

Yeah, it's a real pleasure.

I like both of you guys a lot, and like when my kids fight, I get mad. Over the summer, there was some clip going around about Roman, as he's often want to do, promoting the concerns that I think are legitimate. We should talk about with regard to AI safety, which is term he coined and has certainly done more than anybody on Earth, to bring awareness to.

Simultaneously, Joscha, who's one of the deepest thinkers available, had a little bit of a snarky reply, a comment on Twitter that it's not possible for Roman to simultaneously believe in the simulation hypothesis and also be assured, if I get this correct, of nearly 100% P-Doom. So I hate P-Doom. I think P-Doom is like the redheaded stepchild of the Drake equation. I think it's nonsense. I mean, just like the Drake equation, no one's ever looked at the Drake equation and done an actual measurement that could be falsified. It's just a framework. I think P-Doom is the same. But, Joscha, can you explicate your thesis a little bit more? Why is it simultaneously possible for Roman or anybody else to believe in the simulation hypothesis or why is it a contradiction and also believe in P-Doom? Why is that being high?

Joscha Bach (2:10)

It is totally possible to believe both that we are in a simulation and that we are doomed to a certain degree or to 100 percent because it could be that you're simply in some kind of computer game which I think what the simulation hypothesis comes down to, that we are running in a computer in the basement of a nerd in the parent universe and in this computer game we can be doomed. Of course, this can be the case. It could also be that our actions within the game, despite the game being deterministic and so on, which leads to the usual questions about how we interpret our freedom of will, lead to a certain outcome in which our actions are causing the probability of that doom to change. That's all possible. The question is whether it matters. When you are in a computer game, the thing that you are in is not necessarily the thing that it's a game. You are in there for a certain amount of time. There's ultimately no point to this whole thing.

The question whether you should be worrying whether the AI wins or you win is a different one for most people, I would say, than it is when you believe that we're actually in base reality, and what we do originally matters because our creativity decides on how this universe is going to end up.

Brian Keating (3:24)

Do you think it's possible, Joscha? First of all, let's get this out of the way.

Do you subscribe to any validity, any marginal utility of the concept of P-Doom? Because I don't, but do you?

Joscha Bach (3:37)

Well, I think that it's very difficult to estimate P-Doom reasonably, but we can make the argument if you have different people who see the future differently that this should lead to different probabilities. It might not be that these probabilities are important as a real number, but they are important in relationship to each other in the sense of if you were to shut down the big AI labs, is this going to increase or decrease the probability of this or that event? In this way, I think it makes sense to ask people for how their probability estimates change, even though I think there are too many factors involved to put meaningful even magnitudes on the PDOOM estimates. I sometimes say that my PDOOM without artificial intelligence is higher than this.

That's also a PDOOM estimate, but I mean in the long run, PDOOM is one and long run, We Are All Dead is a famous economist once said, and that's inevitable. But back to this original question, you probably remember the movie The Matrix.