Who Grades the AI Models? | Ben Horowitz & Rayan Krishnan

Rayan Krishnan (0:00)

Every time a new trillion-dollar industry emerges, there's a need for this independent testing group. When Meta released Llama 4 on our held out private benchmarks, the model was actually underperforming. But on all of the major public benchmarks, it was showing incredible capabilities.

Ben Horowitz (0:14)

What's the limit of what you can achieve? And then within that, how are you going about it?

Rayan Krishnan (0:19)

In an ideal world, take a front-tier model and have it train the next version of itself, but obviously that's very expensive and slow. And so what we're doing is forming a set of proxies for every part of the process it takes to build the next version of the models. Evaluations, as they become more complex, have a fewer sample size, but a larger set of criteria or expectations of them.

Erik Torenberg (0:38)

Where do you see the gap that's happening today?

Ben Horowitz (0:41)

The government kind of has an inclination of what it's afraid of, be it biohacking or cyberhacking, but then there becomes the question of, can the model do it and then can you get the model to do it?

Erik Torenberg (0:51)

What do you think the landscape will look like?

SPEAKER_4 (0:54)

AI models keep getting better, but the tests we use to measure them can become obsolete almost as quickly. In this episode, I'm joined by a16z's Ben Horowitz and Jennifer Li for a conversation with Vals founder and CEO Rayan Krishnan about the increasingly difficult problem of measuring AI. We get into why public benchmarks can give a distorted picture of model capabilities, why independent evaluation matters, and what it takes to test a new model in a few hours before it launches.

But this is becoming about much more than model leaderboards. As companies spend more on AI, they need to know which models and agents actually perform best for their own work, and whether that intelligence is worth what they're paying for it. Rayan also explains why benchmarks need to evolve alongside the models, how Vals is measuring recursive self-improvement, and why evals could eventually become a shared language for AI capabilities, risk, and policy.

Erik Torenberg (1:54)

So, I'll start a question from when Vals got starting 2024, after a team discovered that all the public benchmarks are just not sufficient enough to measure model progress, and there needs to be a new methodology and approach coming to keep us on the frontier and help model labs continue to hill climb, take us back to the inception of Vals and what you see was missing in the market then.

Rayan Krishnan (2:18)

Yeah, I mean, so I had a background doing research, in particular building benchmarks and evaluations, and so what was very clear to me was the very tight relationship between what it takes to build new systems for generation, actually new mechanisms for evaluation. In fact, in order to get one, you often need to get better at the other, and actually one of the biggest drivers for model capability is having a new legible way to evaluate models. And so, around early 2024, what you're seeing is there are actually many interesting models coming to market, they weren't all coming from OpenAI, and then also in particular, it was harder than ever to actually ascertain what was newly capable with the new models. And so, in kind of a first principles way, what we realized is that there would need to be some third-party company that solely existed to build really high quality evaluations and benchmarks to be able to discern what was newly possible with these models. And so, we released our first benchmarks in 2024, and now the last couple of years that has kind of been realized by many different parts of the industry.

Erik Torenberg (3:10)

I guess one of the obvious question is, why do you think the labs can't do this by themselves? Because they know the best of where the models are hill climbing on, and what is missing capability wise, why can't they be the benchmarking stores?

Rayan Krishnan (3:23)

Yeah, I mean, internally, they do build a lot of great benchmarks and that's what drives model progress. But I think there's an issue when we speak about model capabilities in a way that's self-reported. And so, one of the early indications of that you saw was when Meta released Llama 4, that was a bit of a disaster. And interestingly, what we saw is that on our held out private benchmarks, the model is actually underperforming. But on all of the major public benchmarks, the questions and rubrics are actually open source. It was showing incredible capabilities. So, there's a huge disconnect between what was self-reported based on these open benchmarks, and then what we were actually finding with our higher quality, higher signal benchmarks. But I think it speaks to a broader concept which the labs, I think understand that they would like to see a rational buying market. They would like to see that when they invest billions of dollars to build a new model, there are actually substantive ways they can point to evidence and say, we're advancing in these ways. And it's not just entirely self-reported to justify that investment. And so you also see instances of DEMIS and others in the industry calling for an ecosystem of third-party evaluators.