HS145: The Economics and Architecture of AI Factories (Sponsored)

Johna Till-Johnson (0:03)

Hi, I'm Johna Till-Johnson. I'm here with my co-host John Burke, and you are listening to Heavy Strategy, the show that tries to ask the right questions, not give the right answers. Joining us today are Thierry Pienaar, worldwide CTO for HPE's HPC and AI Sales Division, and Kaushik Shirhatti, VP of AI Factory at Nvidia, to talk about what it takes to build an AI data center.

And we're talking about how to build an AI data center in-house and why you should consider the HPE and Nvidia AI Factory to get there. So, let's just go ahead and dive right in and kick off with some quick definitions. Kaushik, can you tell us a little bit about what an AI factory is in the first place?

Kaushik Shirhatti (0:44)

Johna, that's a great question to start off with.

So, when you think about a factory, you think about a traditional factory, you think about all the equipment that goes into the factory, right? So, you think about like compute and networking.

When it comes to AI, you think about compute, networking, software, all of the stuff that goes into the factory. And then you start thinking about the inputs and the outputs, right? And so, the way Jensen defined it is you have data going in, you have energy going in, and then you have magic. You have tokens, you have business outcomes, you have intelligence coming out. And so, let me just clarify a couple of things that people can sometimes misunderstand. When people think about these AI factories, people think about these gigantic gigawatt scale factories, which is true. There is a huge infrastructure build-out that we have never seen before. But for a lot of enterprises, it can actually start with an employee having a small DGX spark under her desk, and she has models and she starts producing tokens. That's a mini AI factory. And as the enterprises adopt more AI, they can go into what we have different platforms, like RTX Pro and NBL8 and NBL72, Rack-scale systems that you scale out. The most important thing is that there are enterprise AI factories with enterprise consideration. There are brownfield data centers, air-cooled data centers, and then you have very large AI factories. But the thing that actually really matters is who are these factories for. Right? And I always say that easy is hard. Making something look easy, seamless, effortless is extremely hard. And so what HPE and Nvidia are doing together is that we are making it extremely hard on ourselves to actually help out our customers, which is not, when you think about an enterprise, it's not the CISO, it's not the infrastructure team, it's not the CEO or the CFO. They are all our partners.

Our real customers are the users, agents, and the machines that are harnessing the power of AI to make lives better. It's the banker, it's the doctor who is going to do deep research, oncology research, it's that clinician who can go home early because she's using an AI scribe to her family. Those are the users, those are our real customers. Everybody else is a partner here.

Johna Till-Johnson (3:01)

I love the vivid imagery, but I want to tease out something that's very important. What is the difference between an AI factory and a bunch of chips running an LLM?

What is the AI factory concept here?

Kaushik Shirhatti (3:17)

Yeah, so the concept of AI factory, if you compare to traditional data center, when you mean by traditional chips, like servers running in at high power, the traditional data centers, think about them, were their cost centers. They were doing your enterprise workloads. The AI factories, when you have these factories together, it's essentially a way where you are generating intelligence. When you talk about a token, a token is really a unit of intelligence. If you connect that token to the output, essentially these are revenue generating factories. Now, it's not just about one chip or one system. All of it has to come together. If you think about the simplest analogy I give, is think about a relay race. If you, John and Johna, if you decide to build an Olympic gold winning relay race team, what's the first thing you're going to need? You're going to need four amazing sprinters. Think about Shakira Richardson or Usain Bolt running down the track.

Between HP and Nvidia, we are very blessed because our first sprinter is Compute. All the innovation we are bringing from Nvidia standpoint, from Copper to Blackwell to Vera Rubin now, our CPUs, our networking chips, everything that HPE is bringing in innovation around the hardware around these chips. That's like the heart of the factory. The networking is the backbone. And then you really think about infrastructure and intelligence software.