China Trains Top AI Models With 20x Less Compute | Konstantin Pilz

Konstantin Pilz (0:00)

The China is relying entirely on Western machines to make its AI compute. And if the US wanted to, they could just restrict those exports, and China would be making way less AI compute and would take a much longer time to catch up. And this is also a teaser because next week or very soon, we'll have a whole technical paper analyzing EUV versus DOV and how much China has been importing and how much of a difference this would really make.

SPEAKER_2 (0:24)

All right, we are back. We are live with Konstantin Pilz. He's a co-founder and research fellow at the Center for Technology and Statecraft. Previously, he researched semiconductors and AI policy at the RAND Corporation and the Center for the Governance of AI. Konstantin, welcome.

Konstantin Pilz (0:37)

Thank you. Good to be here.

SPEAKER_2 (0:39)

So tell us about the Center for Technology and Statecraft. There's many AI policy orgs in the world. What makes this one different?

Konstantin Pilz (0:46)

There's many think tanks.

I think maybe the most important thing is that most think tanks focus a lot on engagement and doing near-term policy sprints. DC is very fast moving, even though it doesn't always look like that from the outside. But you need to catch those opportunity windows, and a lot of think tanks do that all the time and go from one sprint to another. I think CTS is an attempt to take a step back and say, okay, what are some of the more fundamental issues we need to solve for AI policy? Then making progress on those by really trying to form a model of the full AI supply chain, and that includes semiconductors, semiconductor manufacturing equipment, the data centers where they are in the world, then making tokens from the chips, and finally the economic impacts and the security impacts of those. So our researchers are really, we're okay to not have as many pieces out per year, but we really want to go very deep and then have a fundamental answer to a hard question in AI policy.

SPEAKER_2 (1:48)

I do notice in mainstream AI safety world, the pacing of the frontier letter came out a couple of months ago. Jacob Cox and reside from Anthropic, what, a week ago? I don't even know.

Konstantin Pilz (1:59)

That was just a week?

SPEAKER_3 (2:01)

Oh my God.

SPEAKER_2 (2:01)

That was about a week ago.

SPEAKER_3 (2:02)

Wow.

SPEAKER_2 (2:04)

Obviously, blew up the zeitgeist, totally raised the salience of AI among the public and among elites, and then the AI safety people rushed to push forward all of these policies about pacing immediately.

It's like, are these really the best policies from a fundamental perspective? Are we not rushing to just do something?

SPEAKER_3 (2:27)

Yeah. I'm also curious how you see it since you go so deep into the actual supply chain.

Konstantin Pilz (2:33)

I think especially on pacing, one question is just like, what is the pace of AI progress in the first place?

How do we know if we have paced? Are we slower now than we have been before? What are the metrics that we care about?

Often, it's actually quite hard to define how we're going to measure AI progress. I think the best way I know of is the Epoch Capabilities Index, the ECI, and just aggregates a bunch of benchmarks. Because before we had that, every benchmark just got saturated and I was like, okay, now I need to, for the next half year, focus on another benchmark. Now we have this which is kind of like a patchwork thing to kind of aggregate a bunch of things and it's not perfect. But at least we know one proxy for how fast AI progress is. But then another issue is how do we know how much progress different actors are making the different companies in the US and then also China. And I think on China, we've really dropped the ball thus far and we don't have good models of China and this includes even fundamental things like how many chips do these Chinese companies have. We have very little data. So this is one of my main projects right now to estimate for each of the top Chinese players, how much compute do they actually have access to? And my estimate is that they have roughly 20 times less compute than the leading AI companies in the US. And I think that it's kind of confusing how they are still training good models, even though they are so far behind on compute. And the only real answer I have to that is that they're using distillation on a very large scale. And they are kind of piggybacking of the US capabilities. And that also means that if you actually wanted to pace, then China would probably be forced to pace too, because they are kind of just relying on US progress to such an extent, at least for a while. Eventually, they will probably catch up.