Will scaling work? [Narration]

**Dwarkesh Patel** (0:00)
Hey everyone, this is a narration of a blog post I wrote called Will Scaling Work? You can find the full version on my website dwarkeshpatel.com. It was originally published December 26th, 2023 Will Scaling Work?
When should we expect AGI? If we can keep scaling LLMs++, and get better and more general performance as a result, then there's reason to expect powerful AIs by 2040 or much sooner, which can automate most cognitive labor and speed up further AI progress. However, if scaling doesn't work, then the path to AGI seems much longer and more intractable for reasons I explained in the post. In order to think through both the pro and the con arguments about scaling, I wrote the post as a debate between two characters I made up, believer and skeptic.
When will we run out of data?
Skeptic. We're about to run out of high-quality language data next year. Even taking hand-wavy scaling curve seriously implies that we'll need 1e+35 flops for an AI that is reliable and smart enough to write a scientific paper. And that's table stakes for the abilities an AI would need to automate further AI research and continue progress when scaling becomes infeasible. Which means we need 5ooms, that is, orders of magnitude, more data than we seem to have. I'm worried that when people hear 5ooms off, how they register it is, oh, we have 5x less data than we need, we just need a couple of 2x improvements in data efficiency, and we're golden. After all, what's a couple of ooms between friends?
No, 5ooms off means we have 100,000x less data than we need. Yes, we will get slightly more data efficient algorithms, and multimodal training will give us more data, plus you can recycle tokens on multiple epochs and use curriculum learning. But even if we assume the most generous possible one-off improvements that these techniques are likely to give, they do not grant us the exponential increase in data required to keep up with the exponential increase in compute demanded by these scaling laws.
So, then people say, we'll get self-play synthetic data working somehow. But, self-play has two very difficult challenges. 1 Evaluation Self-play worked for Ophigo since the model could judge itself based on a concrete win condition. Did I win this game of Go? But novel reasoning doesn't have a concrete win condition. And, as a result, just as you would expect, LLMs are incapable so far of correcting their own reasoning. 2 Compute All these math code approaches tend to use various sorts of tree search, where you run an LLM on each node repeatedly. Ophigo's compute budget is staggering for the relatively circumscribed task of winning at Go. Now imagine that instead of searching over the space of Go moves, you need to search over the space of all possible human thought. All this extra compute needed to get self-play to work is in addition to the stupendous compute increase already required to scale the parameters themselves. Using the 1e+35 flop estimate for human-level thought, we need 9 OOMs more compute atop the biggest models we have today. Yes, you'll get improvements from better hardware and better algorithms, but will you really get a full equivalent of 9 OOMs?
Believer If your main objection to scale working is just a lack of data, your intuitive reaction should not be, well, it looks like we could have produced AGI by scaling up a transformer++, but I guess we're going to run out of data first. Your reaction should be, holy fuck, if the internet was a lot bigger, scaling up a model whose basic structure I can write down in a few hundred lines of Python code would have produced a human level mind. It's a crazy fact about the world that it's this easy to make big blobs of compute intelligent. The sample over which LLMs are inefficient is mostly just irrelevant ecommerce junk. We compound this disability by training them on predicting the next token. A loss function which is almost completely unrelated to the actual task we want an intelligent agent to do in the economy. And despite this minuscule intersection between the abilities we actually want and the terrible loss function and data we train these models with, we can produce a baby AGI, that is GPT-4, by throwing just 0.03% of Microsoft's yearly revenues at a big scrape of the internet. So, given how easy and simple AI progress has been so far, we shouldn't be that surprised if synthetic data also just works. After all, the models just want to learn. GPT-4 has been out for all of 8 months. The other AI labs are only now getting their GPT-4 level models, which means all the researchers are only now getting around to making self-prolay work with current generation models. And it seems like one of them might have already succeeded. Therefore, the fact that so far we don't have public evidence that synthetic data has worked at scale doesn't mean it can't. After all, RL becomes much more feasible when your base model is capable enough to get the right answer at least some of the time. Because now you can reward that 1 in 100 times that the model accomplishes the chain of thought required for an extended map proof, or writes the 500 lines of code needed to complete a full pull request. Soon, your 1 in 100 success rate becomes 10 in 100, then 9 in 100 Now you can try the 1000 line pull request, and not only will the model sometimes succeed, but it will be able to critique itself when it fails. And so on.