Pure Math Breakthroughs Won't Speed Up the Economy | Stephen Hsu

Stephen Hsu (0:00)

If you look at the individual details on every model card or technical report, you'll see the models playing a huge role in building their successor model. And that is RSI. Actually, it could become autonomous. And we made a very beautiful animation for the movie, which shows a model coding up a new version of itself, which is a better, more powerful neural net. And that cycle, once it becomes autonomous, and humans are not in the loop, it may run out of control.

SPEAKER_2 (0:30)

RSI, he was a software entrepreneur, a biotech entrepreneur.

He posts a lot about a lot of different things. Recently, he's been very into AI, and he's working on a documentary called Machine God about recursive self-improvement. Stephen, welcome to MTS.

Stephen Hsu (0:47)

Hey, great to join you again.

SPEAKER_2 (0:50)

Glad to have you. So tell us about what this documentary is about.

Stephen Hsu (0:55)

Well, we shot this thing over about nine months to document the race to ASI happening in the Bay Area, and we interviewed accelerationists who want this to happen as fast as possible, and we interviewed people who are scared to death about it. And I think it's the most sophisticated documentary that's yet been made on this topic.

SPEAKER_3 (1:19)

Yeah. Who are you trying to reach with this documentary? Are you trying to make it hyper-palatable to a wide range of people?

Stephen Hsu (1:27)

We would like it to get broad exposure, but the most important target audience is people with real power in society. Senators, billionaires, CEOs, who are obviously thinking about AI, but there are three key concepts that we want to get across to them that will help them think through what's coming.

SPEAKER_3 (1:48)

What are the three concepts?

Stephen Hsu (1:51)

Let me list them, and then I'll give you a short definition. One is techno capital, which originates from Nick Lann, the philosopher. One is self-improvement, which I think we're going to talk about. The third one is gradual disempowerment.

Very quickly, techno capital is markets are so powerful now, they're going to concentrate resources behind whoever's building the ASI.

There's nothing any one individual can do. If Dario gets hit by a bus or Sam gets hit by a bus, capital is still going to concentrate those resources and the human capital and just keep building. It might happen in the US, it might happen in China, but there's just this irresistible evolution toward superintelligence, and it's just something that people need to understand. RSI is the ability of the models to improve themselves. And in the course of filming the movie, at the beginning in January when we started filming, nobody was really very familiar with the concept of RSI. Now if you look at the details of, as you guys have, I love your show, the creation of the jalapeno chips by OpenAI, individual details on every model card or technical report, you'll see the models playing a huge role in building their successor model. And that is RSI. Actually, it could become autonomous. And we made a very beautiful animation in the movie, for the movie, which shows a model coding up a new version of itself, which is a better, more powerful neural net. And that cycle, once it becomes autonomous and humans are not in the loop, it may run out of control. So that's RSI. And then the third thing is gradual disempowerment, which I think people are intuitively aware of, but usually they've not seen it fully formulated this way.

Every single element of our society, whether you're a corporation, a military, a government, or even just an individual trying to do well in the dating apps, if you have access to a superior intelligence, an AI, that you need to compete against all the other players in whatever game that you're in, you're going to start using that intelligence, that artificial intelligence, and gradually, even if, quote, nothing goes wrong, even if there's no evil AI or there's no AI hacking your systems, you literally are going to surrender power to it because you will get better outcomes. And if you don't surrender power to it, your competitors are going to beat you. They're going to crush you. And that's gradual disempowerment. It's going to happen completely voluntarily as people just hand over power to AI.

SPEAKER_3 (4:29)

Yeah. Something I wanted to bring up was this notion of what can combat gradual disempowerment. And I think a few people have written about this. I think Vitalik has written about this, about how possible human enhancement can be part of DIAQ.

Human enhancement could potentially help us combat gradual disempowerment or gradual obsolescence that AI would pose to us. So I'm curious your thoughts. I know there's definitely counterarguments with like, when would it be taking it too far? But I think you're the right person to ask about this.