Ep 90: AI Pioneer Jürgen Schmidhuber on the State of AI Today

**Jacob Effron** (0:00)
Sounds like you think a crash is coming.

**Jürgen Schmidhuber** (0:01)
There will be one of these stock market crashes.

**Jacob Effron** (0:04)
What about like robotics?

**Jürgen Schmidhuber** (0:05)
Robot hardware is really inferior compared to human bodies, and there's no human-made technology that compares to this app.

**Jacob Effron** (0:13)
It seems a lot more than a hardware problem.

**Jürgen Schmidhuber** (0:14)
You can't have AGI without hardware like that. You can't have AGI just behind the screen.

**Jacob Effron** (0:20)
Are we kind of close to that?

**Jürgen Schmidhuber** (0:21)
It will come, but it will take...

**Jacob Effron** (0:23)
Jürgen Schmidhuber has been set as the father of AI by the New York Times, Forbes and more. He is behind some of the most important advances in the field that really power the AI revolution today. It was a real privilege on Unsupervised Learning to get to sit down with him and talk about everything that's top of mind in the ecosystem today.
We talked about what's missing in models today and what he thinks is required to get artificial scientists that can push them forward. We talked about why he thinks today's CapEx boom is massively overdone, and why he's very optimistic on AI technology, but deeply pessimistic on the model companies, and how he doesn't think recursive self-improvement will actually be a moat for the companies. We also talked about AI safety, and why he's notably less worried than a lot of others in the field. It's just an awesome opportunity to get to sit down with a legend in the field and ask him all these questions. I think folks will really enjoy hearing his perspective. Without further ado, here's Jürgen.
Well, thanks so much for coming on the podcast. Really excited about this.

**Jürgen Schmidhuber** (1:16)
It's my pleasure, Jacob.

**Jacob Effron** (1:18)
Well, I feel like there's so many different things I want to talk about today. You've obviously been a pioneer of a ton of different parts of this current AI moment. I figured where I'd start at the highest level was, as I understand it, you've had this goal for a long time, which was to build an AI smarter than yourself. How close are we to that right now?

**Jürgen Schmidhuber** (1:34)
From a cosmic perspective, we are as close as we were in the 1970s when I first formulated that wish. We are very close, but is it going to be a couple of years or a couple of decades? I'm not totally sure about that because true AI is not just the AI behind the stream, which is working very well and which is passing the Turing test now. True AI is also real robots, real machinery outside of the screen, in the real world, in the physical world, and that's not working as well.
The hardware in the real world has lots of limitations that human bodies don't have, and we still have a way to go to be able to compete with human bodies and physical AI.

**Jacob Effron** (2:27)
Have there been any current AI results over the last few years that have surprised you?

**Jürgen Schmidhuber** (2:31)
Not really. Not to the extent that it surprised people who had no contact to neural networks and artificial neural networks in the previous decades, and suddenly there was a chat CPT moment, suddenly people started being interested in that thing which they had never seen before. So they didn't know that there was a long history of large language models and an even longer history of basic insights and algorithms for training these large language models that goes back to the previous millennium.
So for a guy who was in the center of all of that, it was much easier to predict that than for someone who had totally different interests.

**Jacob Effron** (3:21)
Well, I definitely want to hit on Per cursive self-improvement in meta-learning because I feel like it's been a huge focus of yours for a while. It obviously seems to be a main focus of a lot of the major labs these days. You pioneered a bunch of the research here. How do you articulate where we are today on the path of getting to RSI and what still needs to be solved?

**Jürgen Schmidhuber** (3:41)
In 1987, this was about using meta-evolution, evolutionary programming for evolving better programs that learn to do better kinds of evolution, meta-evolution, I call it. It was very Darwinistic in many ways. Over time, you had better and better learning algorithms, learning to combine code from previous programs in better and better ways to solve problems better and better.
In 1994, we had reinforcement learning techniques, self-referential machines that were able to, they basically were using a universal programming language to generate arbitrary self-modifications of the code that was running the machine, interacting with some environment. And then in 2003, that was a mathematically optimal way of generating self-improvements by a machine that has some software and it's interacting with an environment. And this environment sometimes punishes you or provides a reward, and you want to maximize the sum of all the rewards in your life until the end of your life, and you want to minimize the sum of the pain signals. And then there's an initial software in that machine, and this machine then can, in principle, write programs that modify the initial software. But before it modifies itself like that, it first has to prove, which means there's a proof searcher in the software, it has to prove that the modification that will be caused by the execution of this particular program is useful in the sense that it will lead to more expected reward than the alternative, which would be not to execute this program.