Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI

Amin Vahdat (0:00)

In hardware, again, as you know, there is this opportunity where the more you specialize to a particular workload, the less flexible it is, the faster, the more power efficient the hardware is going to be. So it is this art, and it's this projection of what are you designing to, and how persistent is that workload. In other words, if it's going to go away after a month or two months or three months, even if it's big for those three months, you got a really narrow window to intercept it. So it has to be somewhat durable.

And you have to be able to project exactly what when you can get for specializing to it.

Sonya Huang (0:47)

I'm thrilled to welcome Amin Vahdat to the show. Amin, thank you for joining us for today.

Amin Vahdat (0:56)

It's really exciting to be here. I'm really looking forward to it.

Sonya Huang (0:59)

I am very excited for today's topic because we are in the middle of the biggest CapEx buildout in human history. You are in the middle of it. Google alone is expected to spend more than $200 billion on CapEx this year, most of which is going into building data centers. You're at the center of it all. You were named the head of Google's AI infra at the end of last year.

So you are the man spearheading the efforts of one of the most capital-intensive buildouts in human history, and so I am really excited to get into it with you today.

Amin Vahdat (1:30)

It's a huge year for sure, of course, across the industry, including at Google. Really, I don't think we've ever seen anything like this, honestly. Certainly not at Google, but as you said, I think in the history of humanity, in terms of the buildouts and the pace of transformation, really unbelievable.

Sonya Huang (1:46)

Before we get into it, maybe just crash course for the audience. What is an AI data center and how is it different from a non AI data center?

Amin Vahdat (1:55)

It's a very good question in that AI data center and a non AI data center do have a lot of similarities, actually, and they're not radically different. I mean, it consists of concrete, an enclosure. It consists of electrical yards, mechanical yards, cooling. You have row after row of power being distributed. There's a huge amount of network infrastructure there. In other words, we connect large amounts of computes to one another.

A fair amount of storage infrastructure goes in there. I think that the big difference that we're seeing with AI infrastructure is specialization. In the past, when we're building a data center, it really is a 20, 25, 30-year building investments. And we're thinking about how it's going to be evolving over that 20, 25, 30-year period. You might have servers go into it. You might have networking storage. You might have some accelerators, GPUs, TPUs, whatever they might be going into it. But it's a 25, 30-year planning horizon.

And the lifetime of the hardware might be six years. So many generations that we have to plan for. An AI data center oftentimes is going to be more purpose-built. So in other words, we are oftentimes co-designing the building with the hardware that might go into it. We might be saying, you know what, actually, we're not going to put a lot of storage into this building. Why? Because a storage rack might have 10, 20, 30, 40 kilowatts of power. You put that next to a TPU rack or a GP rack that easily is hitting hundreds of kilowatts today.

Might hit megawatts. People are talking about that in the next few years. Designing a building that might take 30 storage racks in a row versus one or two AI racks in that same row. Very, very different design. Just think of it in terms of the size. Think of it in terms of the power and how that would be distributed across the building, etc.

Networking would be the same. The amount of networking that you would need for a storage rack, tiny, especially if it's hard drives, compared to an AI rack. So if you're trying to make it fully fungible, you'll probably make it too big and too overbuilt. Fungible over certainly a 30 year period. An AI data center likely is going to be much more purpose built, co-designed with the hardware, even to the point of cooling and power distribution, etc. So a lot more co-optimization.

Sonya Huang (4:27)

You all delivered a big vera-rubin cluster to one of my portfolio companies, Ineffable Intelligence. And I saw the photographs of it as it went out. That is a thing of beauty.