Ilya Sutskever — We're moving from the age of scaling to the age of research

**Ilya Sutskever** (0:00)
You know, it's crazy that all of this is real. Yeah, don't you think so?

**Dwarkesh Patel** (0:06)
Meaning what?

**Ilya Sutskever** (0:06)
Like all this AI stuff and all this Bay Area, yeah, that it's happened, like, isn't it? Straight out of science fiction.

**Dwarkesh Patel** (0:13)
Yeah, another thing that's crazy is like how normal the slow takeoff feels. The idea that we'd be investing 1% of GDP in AI, like I feel like it would have felt like a bigger deal, you know, where right now it just feels like.

**Ilya Sutskever** (0:27)
We get used to things pretty fast turns out, yeah.
But also, it's kind of like it's abstract, like what does it mean? What it means that you see it in the news, that such and such company announced such and such dollar amount. That's all you see. It's not really felt in any other way so far.

**Dwarkesh Patel** (0:45)
Now, should we actually begin here? I think this is an interesting discussion. Sure. I think your point about, well, from the average person's point of view, nothing is that different will continue being true even into the singularity.

**Ilya Sutskever** (0:57)
No, I don't think so.

**Dwarkesh Patel** (0:58)
Okay, interesting.

**Ilya Sutskever** (0:59)
So, the thing which I was referring to not feeling different is, okay, so such and such company announced some difficult to comprehend dollar amount of investment.

**Dwarkesh Patel** (1:12)
Right.

**Ilya Sutskever** (1:12)
I don't think anyone knows what to do with that.

**Dwarkesh Patel** (1:14)
Yeah.

**Ilya Sutskever** (1:15)
But I think that the impact of AI is going to be felt. AI is going to be diffused through the economy. There are very strong economic forces for this.
And I think the impact is going to be felt very strongly.

**Dwarkesh Patel** (1:30)
When do you expect that impact? I think the models seem smarter than their economic impact would imply.

**Ilya Sutskever** (1:38)
Yeah. This is one of the very confusing things about the models right now. How to reconcile the fact that they are doing so well on evals. And you look at the evals, and you go, those are pretty hard evals, they are doing so well. But the economic impact seems to be dramatically behind. And it's almost like, it's very difficult to make sense of how can the model on the one hand do these amazing things, and on the other hand, like repeat itself twice in some situation. An example would be, let's say you use Vibe coding to do something, and you go to some place, and then you get a bug. And then you tell the model, can you please fix the bug? And the model says, oh my god, you are so right, I have a bug, let me go fix that. And it introduces a second bug. And then you tell it, you have this new, the second bug. And it tells you, oh my god, how could I have done it, you are so right again, and brings back the first bug. And you can alternate between those. And it's like, how is that possible? It's like, I'm not sure. But it does suggest that something strange is going on. I have two possible explanations. So here, this is the more kind of whimsical explanation, is that maybe RL training makes the models a little bit too single-minded and narrowly focused, a little bit too, I don't know, unaware, even though it also makes them aware in some other ways. And because of this, they can't do basic things. But there is another explanation, which is, back when people were doing pre-training, the question of what data to train on was answered. Because that answer was everything. When you do pre-training, you need all the data.
So, you don't have to think, is it going to be this data or that data. But when people do RL training, they do need to think. They say, okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing. And from what I hear, all the companies have teams that just produce new RL environments and just add it to the training mix. And the question is, well, what are those? There are so many degrees of freedom. There is such a huge variety of RL environments you could produce. And one of the one thing you could do, and I think that's something that is done inadvertently, is that people take inspiration from the evals. You say, hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task, right? I think that is something that happens and I think it could explain a lot of what's going on. If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real world performance, which is something that we don't today exactly even understand what we mean by that.