Roman Yampolskiy: AI Can’t Be Controlled — and We’re Building It Anyway

**Brian Keating** (0:00)
A computer scientist who helped found the field of AI safety just told me we're building something we can never switch off and that the smartest people in the room agree with him.

**Roman Yampolskiy** (0:09)
We're going to have systems ten, hundred, thousand, million times smarter than us. What does that mean? They see patterns we don't see. You can have squirrels, monkeys, whatever you want. They are very intelligent beings, but they're not competitive with us. We're just on a different level. And it's exactly what we're going to see here. Having an agent and giving it full access to your computer, bank accounts, your email, sounds like the dumbest thing you can possibly do. And watching smart people do that really blows my mind. We're asking for a pause in frontier model development contingent on someone solving control. If I'm right and control is unsolvable, that moratorium becomes a permanent ban. If I'm wrong, in 10 years, I'll get utopia, free stuff, and be very happy to be wrong.

**Brian Keating** (0:52)
That's Roman Yampolskiy, a computer scientist who helped found AI safety and who now argues that that can't ever be done. Not hard, provably impossible. Now, I'm used to going into the impossible, but today he's going to get into why the math said so. For the first time at this level of detail and the one button he'd actually press if he had the opportunity. We can't bound the limits of knowledge. And part of controlling super AI, again, I'm being devil's advocate here. I'm not saying this is my position. I'm just saying this is what I think David, past guest David Doerrsch would say, is that because to control something, you need to have knowledge of its future prediction, of its future behavior. Although you can't have perfect knowledge of anything, right? That's impossible for sure. But the question is, can you have the same level of controllability via the knowledge that a human who's a universal explainer can glean? And I guess he's saying, you're saying it's impossible for a human to ever control AI, but he's saying humans are universal explainers, therefore nothing explainable, even with errors and stochastic notions of what explanation means, is not fundamentally restricted. So it does seem like you guys don't agree and that's fine. That's not a formal proof, by the way. I'm just saying, he's saying humans can explain things, AI is something that could be explained, therefore it's not impossible to explain them. But I think you would say it is impossible to control them because you cannot explain them, correct?

**Roman Yampolskiy** (2:14)
So there is theory and practice. In theory, given infinite time, average human with pencil and paper can probably figure out everything. We don't have infinite time and we have limited size brain with limited size memory cells. Our ability to survey information is limited. So if you had a mathematical proof and it was a billion pages long, no human can verify that proof.
In theory, they could, but in practice, it's not going to happen.

**Brian Keating** (2:42)
I thought you were going to say, in theory, communism works, but in practice.

**Roman Yampolskiy** (2:47)
If we have a real world situation where you have a system, let's just say it's just like humans, but it's a billion times faster. We're not competitive in that environment. We simply don't have time to react, to do anything whatsoever, to counteract what the system is going to do, but we are not equal. We're going to have systems ten, hundred, thousand, million times smarter than us. What does that mean? They see patterns we don't see.
I always bring up examples from animal kingdom, right? You can have squirrels, monkeys, whatever you want. They are very intelligent beings. Eventually they have some language, culture. They're starting to use tools, but they're not competitive with us. We're just in a different level. And it's exactly what we're going to see here.
Him being a very good professor at a very good university, he understands and he selects students. He doesn't select them at random because all humans are universal explainers. He looks for the one with highest intelligence.
Why? Because they're going to finish things in time. They're going to understood and publish on time. And it's exactly the same. Instead of looking for a student with IQ of 130, now you're comparing machines with IQ of 1000 to humans and saying, well, technically they all in the same class of automata. They all touring complete. True, but it means nothing for safety.

**Brian Keating** (4:08)
Einstein said the following, Roman. He said, no problem can be solved from the same level of consciousness that created it. So, did Einstein kind of predict some of the claims that you're making now that we have basically have been from the start unable to even grapple with the questions of what sort of entities we're creating?