Superintelligence Strategy (Dan Hendrycks)

Dan Hendrycks (0:00)

Compared to nuclear weapons, I think it's harder to make cutting-edge GPUs, given a billion dollars. I mean, you can't do it with a billion dollars. If it's $10 billion, you can't do it. There was Situational Awareness by Leopold Aschenbrenner, which was arguing for something like a Manhattan Project for developing AGI and super intelligence before China. So it's basically, let's beat China to the punch, get super intelligence, prevent them from building it. The West will dominate the world.

SPEAKER_2 (0:28)

You know, Eliezer got in trouble in Time Magazine when he spoke about bombing data centers. There was a big hoo-ha at the time. You used the words kinetic strikes.

Dan Hendrycks (0:36)

We discussed kinetic attacks in the escalation ladder. So there's many ways to disrupt projects. You could do cyber attacks on them. You could do some gray sabotage. There's hacking to sort of poison their data or sort of make their GPUs not function as reliably or threaten to use force. But I don't think that those are really necessary. So like that would be an escalation ladder. But if the US is on top of this, they don't need to resort to that.

SPEAKER_2 (1:12)

Today's episode is sponsored by Prolific. This is Sara Saab, who's a VP of product at Prolific. Sara is using the Prolific platform to understand how large language models think by designing the next generation of benchmarks with human participation.

SPEAKER_3 (1:29)

We study the benchmarking based on the demographic stratification of the humans doing the evaluations. So you can see stuff emerge in the data, like people of this age range think this model is better on helpfulness, but people of that age range disagree.

SPEAKER_2 (1:48)

Go to prolific.com.

SPEAKER_4 (1:52)

This podcast is supported by Google. Hey folks, Taylor here, creator of Gemini CLI. We designed Gemini CLI to be your collaborative coding partner at the command line. Think about those tedious tasks like fixing a tricky bug, adding documentation or even writing tests. We built Gemini CLI to handle all that and more. It iterates with you to tackle your most challenging problems. Check out Gemini CLI on GitHub to get started.

SPEAKER_2 (2:21)

I want to spend most of the show talking about your very interesting new super intelligence strategy paper, which you fairly recently published. So maybe starting with Humanities last exam. What's the story behind that?

Dan Hendrycks (2:33)

The MMU data set, which I made as a graduate student some years ago, was getting saturated. It seemed that pretty much all of the evaluations were getting saturated. So people didn't really know what was going on with AI capabilities. So there seemed to be particularly strong reasons for people just being informed about developments to develop something new.

So I was also experiencing that people, experts don't really have data sets in them. You can't just hire a few experts and they can come with a data set. They don't have enough complicated ideas in them. However, I think individual experts might have a question in them instead. So with Humanity's last exam, the idea was we have a global effort with various post-docs and professors, each contributing a question or a few questions to stump existing AI systems. These are questions that they would find very difficult to answer. They would find it impressive if the AI systems could answer them. This would be approximating the human frontier in some sense, of knowledge and reasoning for closed-ended questions where we already know the answer. We did that for some months and we got several thousand questions out of that. I think this will be a good tracker for whether AI systems can automate a lot of the theoretical parts of sciences, whether they can solve difficult analytic questions. Not experimental, like it doesn't test its ability to run biology experiments. That's more motor skills among other things, or requires motor skills, but for things that are more mathematically related, or require some very complex reasoning, that's what this captures. So I think that when it's solved, it's roughly an end of a genre, or near the end of a genre of asking it closed-ended questions, for which there are objective answers, it seems like that would be toward the end of it. I think that individual problems that it would solve in the future would be interesting enough to be like papers in their own right. So once we get through this set of questions, the types of problems it could tackle would be like questions that would be worthy of their own paper, like it solved a conjecture, for instance. So it is tracking the ability level up to the point where individual questions themselves are very interesting, instead of just the data set.