Why Traditional Benchmarks Fail Modern AI Models with OpenAI Research Scientist Noam Brown

**Noam Brown** (0:00)
With GBT-3, you couldn't scale test time compute. Like if you gave it a budget of $10 million and said, okay, well, let's see what GBT-3 can do. It really can't do that much. The Procureness Frameworks and responsible scaling policies, they don't really account for the amount of test time compute. They just say, okay, well, what's the capability of the model? The problem is we're in a world now where the capability of the model is a function of how much money you put into it, basically. If you give it a budget of $10,000, it can do a lot more than what it can do with a budget of $10.
Give it a budget of $10 million, it can do even more. At what budget should you evaluate these models? The policies that exist today don't really address that question.

**Sarah Guo** (0:43)
Hi, listeners. I'm Sarah Guo, and welcome back to No Priors. Today, I'm here with Noam Brown, one of our godfathers of AI reasoning. We talk about the broken state of evaluations, very large scale test time compute, how he thinks about recursive self-improvement, and what's next on the horizon for competition at the frontier. Welcome.
Noam, I'm so excited to have you back.

**Noam Brown** (1:05)
That's great to be back, yeah.

**Sarah Guo** (1:06)
You are our first guest. I'm very proud of my taste in friends and researchers for the pod, given how important inference time scaling has become to the industry. You should be proud too, having actually pioneered it.

**Noam Brown** (1:21)
Played a part, yeah, among many others.

**Sarah Guo** (1:24)
You just wrote this essay that really resonated about large-scale test time compute and why the industry is not evaluating these models as robustly as it should be. What was the motivation for it?

**Noam Brown** (1:37)
Yeah, the motivation was we released 5.5, and the initial reaction was kind of skepticism that it was a substantially better model. To be fair, that only lasted for a few hours before people had some time to play around with it and tried it out themselves and they saw that it was actually substantially better. But I think a lot of the skepticism came from the benchmark grid that was published. Basically, whenever a new model is released, there is this benchmark grid where they show all these different benchmarks on the x-axis and then the performance of different models on the y-axis, and you can just compare different models. It's like a single number for a model on a single benchmark. And if you look on paper at the difference between 5.5 and 5.4 or other models, it was an improvement but it wasn't a huge improvement. It was only a few percentage points in some benchmarks. So people looked at that and they were skeptical that it was actually a better model. Once they played around with it, the story changed. I think the reason why it doesn't show up as so much better on the benchmarks is because the benchmark results are being presented in the wrong way.
They're not controlling for the amount of test time compute that is being used on that benchmark question. It turned out that 5.5 is just much more efficient with its thinking. If you run it at max settings, 5.4 is thinking for a lot longer. It takes longer to get back a response than 5.5.
Once you control for the amount of thinking time, actually you can see that 5.5 is a substantial jump over 5.4. That is I think people's day-to-day experience with it. Then when I mentioned this to people, the reaction, the typical question I get is like, well, why not just have 5.5 think for as long as 5.4? And the question is like, well, how long should they think for? Typically, the response I get is, well, until the performance plateaus.
There's at some point where the performance on the benchmark is going to plateau and you just evaluate to that point. The thing is, the point at which it plateaus is actually really far out these days. I mean, it's true in GPD 3 land back in 2022, the models couldn't really think productively for that long. And so you could just run them until they plateau. It's not that far away.
But what we're seeing today with the modern models is that 5.5 and other models can think for, if you scaffold them reasonably well, can think for weeks even before having performance plateau on some of these benchmarks. And so the point at which they plateau is simply too far out to reasonably test.

**Sarah Guo** (3:52)
We all need to actually reinforce either a patience limit or a budget limit from a token perspective now. And that wasn't true a few years ago.