Noam Brown – Agent swarms, alignment, & recursive self-improvement

Dwarkesh Patel (0:00)

Today, I'm chatting with Noam Brown, who is a researcher at OpenAI. He was one of the foundational contributors to what became O1 and the reasoning models, and now he's working on multi-agent systems. Speaking of which, you guys announced last week that you solved one of the Millennium Prize problems with a system of 10,000 different AI agents that spent 130 billion tokens over 88 hours. One of the reasons I was interested in talking to you, is I think you were one of the first people, maybe two or three years ago, who was thinking about how the reasoning models would allow us to see into the future, because if you scale up inference compute, you can see what the base capabilities of the models will be a few years in the future. I feel like you're in a similar position now to help us understand what future capabilities will look like, given the enormous scaling of agent sizes that we can do right now.

Noam Brown (0:48)

So the way I think about it, when you plot the performance of these reasoning models with test time compute on the x-axis and performance on basically any reasoning benchmark on the y-axis, you see a very clear pattern where the longer these models take to think about their answer, the better they do. This is like a very natural thing. It's the same thing with people. If you're taking the SATs, you have five minutes to go through the entire exam, you're not going to do very well. If you have five hours, you're probably going to do a lot better. The AI models are pretty similar and they'll spend that time doing this monologue to themselves, figuring out, going through different cases, ruling out different possibilities, building on some of their previous discoveries. The problem is that as you push out further and further, you hit a latency bottleneck. You don't want to sit around for three years waiting for a response. So, what you can do is what a lot of people do is they paralyze, they just get a team of people.

If you're going to found a company, you want to get a group of people together so you can go faster. It's the same thing with these AI models that it helps to just have multiple agents working on something because they can just go faster.

So, multi-agent is a way of scaling test on compute in parallel instead of purely serial. It is less efficient because it doesn't have, it's not like a single agent has all the context to itself, but it is a very effective way of scaling test on compute if it's done well.

Dwarkesh Patel (2:08)

Okay. I'm going to ask a bunch of naive questions because these systems, so this is an unreleased model, so we haven't publicly seen how these systems work. So, I just have a bunch of ways in which I'm confused about what the qualitative properties of such systems are.

I am shocked by the scale of cognitive effort that you can concentrate in such a short period of time. So, if you think about what 130 billion tokens are, if it was a single human thinking as a full-time job, stretched back to back, 130 billion tokens would be a human thinking for 4,000 years, eight hours a day or something, working a normal work week. So, starting from like ancient Sumeria up till today, a single sequential human thinking that long, concentrated in 88 hours, I feel like qualitatively that is a super important consideration, and I'm surprised that there isn't a bigger parallelization penalty, that you can just have 10,000 agents collaborate, and because maybe the agents are better collaborating than humans might be, they're going much faster that they can actually productively collaborate at such a big scale. Or maybe they, I don't know, maybe there is a big parallelization penalty.

Noam Brown (3:13)

Yeah, let's talk about the parallelization penalty, and we can talk about the qualitative stuff. Because the truth is that we don't have very good science on multi-agent scaling up to this kind of scale. Yeah. So when we released 5.6, I think that was the first time that we had a proper multi-agent system in our models, and we actually did in the blog post show some plots of the scaling performance of multi-agent systems because we have it as an option. It's ultra mode, and the default is four agents, but you can set that to higher. In the plot, we show, okay, here's what the performance looks like on some benchmarks for one agent, for four agents working together, for 16 agents working together. And what you see, and it depends on the benchmark, but for some of the benchmarks, basically if you have four agents working on the problem, it is done twice as fast.