China Is About To Catch The US In AI (Here's The Timeline) | Justin McAfee

SPEAKER_1 (0:00)

College football is back, so Hilton called on me, the superstition concierge, to make your fan rituals a reality. Need a room to match your lucky number? We got you. Wanna make sure our team doesn't wash your lucky jersey? Ooh, that smells lucky.

Hilton's unmatched hospitality can keep up with any superstition. Even a marching band wake up call at 555 in 55 seconds. Hit it! When you need a team that will do whatever it takes on game day, it matters where you stay. Hilton, for the stay.

SPEAKER_2 (0:30)

When you need to build up your team to handle the growing chaos at work, use Indeed Sponsored Jobs. It gives your job post the boost it needs to be seen and helps reach people with the right skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this show will get a $75 sponsored job credit at indeed.com/podcast. That's indeed.com/podcast. Terms and conditions apply. Need a hiring hero? This is a job for indeed sponsored jobs.

Justin McAfee (1:00)

China produces 2x the electricity of us. They're adding grid capacity six times faster than us. Their AI energy usage demand is like one to five percent of the capacity they've added in the past five years, whereas in the US it's like 50 to 70 percent of the capacity we've added.

Anthony Pompliano (1:17)

What's going on, guys? Today we have a great conversation with Justin. He's one of the AI researchers at Delphi Digital. In this conversation, we're going to get deep into the weeds. What is going on at the American close-sourced AI model labs? Why is China putting so much pressure on these labs? How's it changing? What you and I pay when we actually access these models? On top of that, what are the constraints that the Chinese models are facing and how is that constraint leading to innovations that are actually making the US labs a little nervous? On top of that, we talk about some of the American challengers and how it's changing the entire industry, both for software and hardware, and ultimately how maybe you and I are going to end up being the big winners because competition breeds excellence and a better product at better pricing for everybody. This conversation is fascinating. I think you're going to get a ton out of it. If you're into Bitcoin, AI, or any frontier technology, you're going to want to be paying attention to what we talk about in the next 30 minutes or so. Here's my conversation with Justin.

Justin, everyone's been talking about the American closed-sourced AI labs. They've been doing a fantastic job of bringing products to market, but all of a sudden, there's a challenger that's come on the horizon in the Chinese AI labs. One of the things I find interesting is they have certain constraints that maybe the AI labs don't have, but constraints can breed innovation. It looks like that's what's happening with the Chinese AI models. You've recently written a great research report explaining what some of these innovations are. Can you just walk us through what the constraints the Chinese labs have faced and then what innovations that's now leading them to have on a breakthrough front?

Justin McAfee (2:44)

Yeah, for sure.

With the US, we started expert controls back in October of 2022, banning the A100 and H100 class GPUs. From there, we did some additional tightening as well.

What this essentially did is put China in a situation where they had limited compute, limited memory, and half the chip-to-chip bandwidth that US labs face. This resulted in a situation where they needed to innovate across different levels of the stack in order to successfully serve their models on their cheaper chips. In the report, I discuss, there's three main innovation themes. The first is the training and systems efficiencies, and this is dealing with that lower chip-to-chip bandwidth. Then there's the architectural efficiencies that they've done, which is how essentially you route around the lower memory constraints. Then one of the large areas, I would say, is on the post-training efficiency, which is taking a lot of the compute-intensive pre-training phase and shifting those capability gains into the post-training phase. So yeah, they've had to successfully, they've had to deal with these constraints overall, and it's actually what's pushed them to be so innovative and so cost-effective to be able to compete against our frontier labs today.

Anthony Pompliano (4:12)

When you look at these different levels of innovation, is there one that you point to where you're like, this is the one that really matters, or is it actually the fact that they've done it on all three layers that is leading to the performance gains that they've had?