World Models, Explained

**Ankit Gupta** (0:00)
One of the biggest open problems in AI right now is how to solve sample efficiency. That is, how do you get models to quickly learn new tasks or skills from relatively small amounts of training data?

**Francois Chaubard** (0:08)
Humans do this incredibly well. We can learn new games, concepts and skills, often after just a handful of tries. Our best models, on the other hand, often need tens of thousands of data points just to learn.

**Ankit Gupta** (0:18)
So today, we're gonna discuss what many top researchers believe is the most promising path to closing that gap, world models.

**Francois Chaubard** (0:24)
We're gonna discuss the motivation and math behind world models, current applications, and why this approach might be the key to unlocking AGI.

**Ankit Gupta** (0:39)
You and I have talked a lot about the various ways people are training models and the sample efficiency of them. Why don't we start by just defining sample efficiency and how we intuitively think about it as humans?

**Francois Chaubard** (0:49)
Yeah. So I think from my perspective, the two major problems that we have left to solve is intelligence per watt and intelligence per sample.
Intelligence per watt is like how many valve perplexity points we get per watt of spend. And then intelligence per sample is basically if I have one additional sample in my dataset, how much more intelligent am I getting? And so if I imagine I have a new tasks like RKGI, for example, I think like really Francois Chalet has been on the forefront of this thinking and talking about intelligence as a rate of skill acquisition versus skill acquisition. And that's very different. And so how fast do we get smarter with more and more samples? And these things are incredibly poor at getting smarter with fewer and fewer samples.

**Ankit Gupta** (1:33)
And for context, the RKGI test sets are a really good example of cases where humans are intuitively very good at them. Most humans can intuitively solve those puzzles with some amount of thinking and effort. But our current state of the art AI systems, what people consider frontier intelligence, basically can't do them.

**Francois Chaubard** (1:51)
Right. I mean, we come into new problems with such inductive bias from K through 12, like all these math and school that we've had, that these models are getting from compressing the entire Internet.
When we come in, we're not coming in Tabula Rasa just bare bones. But even so that they have, I don't know what percent of the Internet you've read. I've read very little percent of the Internet. But despite that, and having read the entire Internet, it still can't really do well in generalizing to these new tasks.

**Ankit Gupta** (2:23)
Now, let's think about this in the extreme cases. In the extreme case where, let's say, we were perfectly sample-efficient. We were as sample-efficient as possible. What would that mean in terms of a model that is taking a set of actions in the world?

**Francois Chaubard** (2:37)
Well, I guess the perfect sample efficiency would be zero samples. And there are examples of this. And that sounds absurd to say. But the example, the hypothetical I'll give on this is, imagine I had a perfect world model, then I should never go to the environment to go and collect samples to train on. And well, that can't possibly happen.
No, it actually can happen. We do it all the time. It's called Newton's Second Law of Motion. It's like Newton mechanics, we basically know how to get an object from point A to point B with a rocket quite easily, just by following Newton's Laws of Motion.

**Ankit Gupta** (3:14)
Yeah, like when NASA plans to intercept an asteroid and is planning it years in advance and can set it off in a trajectory where it just glides to the right thing and intersects to the right point, that is an example of a perfect world model we've built where we're then just letting that world model act. And that system does not need to intelligently collect new samples from the environment to decide which direction to go next. It's already been pre-programmed and can perfectly do it.

**Francois Chaubard** (3:39)
Yeah, can you imagine if we needed to collect one million training examples of us shooting spaceships to the moon to know how to do it? But we definitely wouldn't have the Apollo missions, right? But we do have that ability because the real world is differentiable and we can do something called model predictive control that we're going to talk about in a little bit. But even in our own brain, I was just thinking about this on the drive up. But there's so many ways that I can basically think about the things that you are going to say or what VC is going to say when I was pitching them or what a customer might say.