Into The Impossible: Can Superintelligent AI Be Controlled?

SPEAKER_1 (0:01)

When power, prediction, and human survival collide, the conversation gets intense fast. In this episode of Into The Impossible, Brian Keating brings back Joscha Bach for a second appearance and Roman Yampolskiy for a third, and these two dive straight into the most disputed question in AI: is superintelligence a tool we can manage, or a force that will outrun us?

Right from the start, Brian presses them on the tension around AI safety, simulation theory, and P-doom—the probability that advanced AI leads to catastrophe. And here's where it gets fascinating: Joscha says there's no contradiction in believing both in a simulation and in doom. His reasoning? You could be in a computer game where "we are doomed" no matter what. But Roman completely rejects that framing. He says, "Pain is pain and love is love; that is the only life I have." For him, P-doom isn't some philosophical game—it's a practical risk estimate. He actually compares control of superintelligence to perpetual motion. If something has essentially zero chance of working, we shouldn't pretend otherwise. That's a strong stance. And he goes further, insisting that AI isn't just a tool, but a class of agents. He says, "You cannot get consent from eight billion humans because they do not understand what is coming."

Then Brian brings up the Trinity test and the fear of igniting the atmosphere, asking whether earlier scientists should have stopped nuclear development.

Roman's response is really interesting. He says that analogy fails because nuclear weapons are tools, while advanced AI systems are decision-makers.

The danger isn't bad intentions alone—it's that a smarter system can pursue goals that simply collide with human survival. Now, Joscha pushes back with more nuance. He says the nuclear analogy is limited but not useless, and recalls growing up in the 1980s with fears of destruction that never fully came to pass. He's more optimistic about today's models because they seem to scale with compute, which suggests a degree of controllability. Building on that point, Joscha draws a key distinction. He separates narrow AI, like self-driving systems, from human-level AGI and from recursive self-improvement. He argues that today's large language models may be more like "deep fakes of human intelligence" than independent superminds. Roman warns, though, that once AI systems can automate scientists and engineers, the process can become self-accelerating and irreversible. The debate then shifts to regulation and power. Roman says major labs may support regulation not only for safety, but also as a barrier to entry that protects their position. He wants decentralized AI that empowers ordinary people.

The final stretch moves into consciousness, morality, and even God. Roman says consciousness is irrelevant to safety, while Joscha treats it as a computational question. But both return to the same core issue, not whether intelligence exists, but whether it can be directed. As Brian frames it, the real challenge is simple and frightening—"can you control something a million times smarter than you?"