The Best of 2024 with Sarah Guo and Elad Gil

**Elad Gil** (0:05)
Hi, No Priors listeners. I hope it's been an amazing 2024 for you all. Looking back on this year, we wanted to bring you highlights from some of our favorite conversations. First up, we have a clip with the one and only Jensen Huang, CEO of Nvidia, the company powering the AI revolution. Since our 2023 No Priors chat with Jensen, Nvidia's tripled in stock price, adding almost 100 billion of value each month of 2024 and entering the $3 trillion club. More recently, Jensen shared his perspective again with us, this time on why Nvidia is no longer a chip company but a data center ecosystem. Here's our conversation with Jensen. Nvidia has moved into larger and larger, let's say like unit of support for customers. I think about it going from single chip to server to rack and VL72. How do you think about that progression? Like, what's next? Like, could Nvidia do a full data center?

**Jensen Huang** (0:56)
In fact, we'd build full data centers. The way that we build everything, unless you're building, if you're developing software, you need the computer in its full manifestation. We don't build PowerPoint slides and ship the chips. We build a whole data center. Until we get the whole data center built up, how do you know the software works? Until you get the whole data center built up, how do you know your fabric works and all the things that you expected the efficiencies to be? How do you know it's going to really work at the scale? That's the reason why it's not unusual to see somebody's actual performance be dramatically lower than their peak performance as shown in PowerPoint slides.
Computing is just not what it used to be. I say that the new unit of computing is the data center. That's to us.

**Elad Gil** (1:54)
So that's what you have to deliver.

**Jensen Huang** (1:55)
That's what we built. Now, we build a whole thing like that. And then we, for every single thing, every combination, air-cooled, x86, liquid-cooled, grace, ethernet, InfiniBand, MVLink, NoMVLink, you know what I'm saying? We build every single configuration. We have five supercomputers in our company today. Next year, we're going to build easily five more. So, if you're serious about software, you build your own computers. If you're serious about software, then you're going to build your whole computer. And we build it all at scale. This is the part that is really interesting. We build it at scale and we build it vertically integrated. We optimize it, full stack, end to end. And then we disaggregate everything and we sell it in parts. That's the part that is completely utterly remarkable about what we do. The complexity of that is just insane. And the reason for that is we want to be able to graft our infrastructure into GCP, AWS, Azure, OCI. All of their control planes, security planes are all different. And all of the way they think about their cluster sizing, all different.
But yet, we make it possible for them to all accommodate Nvidia's architecture so that CUDA could be everywhere. That's really in the end, the singular thought, that we would like to have a computing platform that developers could use that's largely consistent. Modulo 10% here and there because people's infrastructure are slightly optimized differently. And Modulo 10% here and there, but everything they build will run everywhere. This is kind of one of the principles of software that should never be given up, and we protect it quite dearly. It makes it possible for our software engineers to build once, run everywhere, and that's because we recognize that the investment of software is the most expensive investment. It's easy to test. Look at the size of the whole hardware industry, and then look at the size of the world's industries. It's $100 trillion on top of this $1 trillion industry, and that tells you something. The software that you build, you basically maintain for as long as you shall live.

**Elad Gil** (3:59)
We of course have to mention our conversation with the lovely Andrej Karpathy, where we dig into the future of AI as an exocortex, an extension of human cognition. Andrej, who's been a key figure in AI development from OpenAI to Tesla to the education of us all, shares a provocative perspective on ownership and access to AI models, and also makes a case for why future models might be much smaller than we think. If we're talking about a exocortex that feels like a pretty fundamentally important thing to democratize access to, how do you think the current market structure of what's happening in LLM research, there's a small number of large labs that actually have a shot at the next generation progressing training, how does that translate to what people have access to in the future?