The AI Energy Stack: The Actual Industry Winners

**Josh** (0:00)
For three years, the AI trade has been one simple idea. Buy the chips. Nvidia became the most valuable company on earth by making these chips, and they've made countless of investors rich because of this. But this investment thesis now has become overcrowded. Not because AI is slowing down, but because there's another problem that has surfaced. No one can turn these GPUs on. Trillions of dollars being spent in AI, and all these GPUs are collecting dust in data centers. The problem, energy, electrical grids, wiring, networking, the transformers, the web of infrastructure that sits around a GPU, that allows it to keep alive, to talk to each other, to transmit petabytes of data between each other. This is the next constraint that hasn't been solved yet, and where the majority of the AI capital will eventually flow. Now, there's four physical things that keep a chip alive, essentially the power to run it, the light to transmit all the data between them, the silicon wiring between them all, and then somewhere to rent it all from. Almost nobody is really talking about this. And on this episode, we're going to unpack that specific layer of the infrastructure stack, the power layer, and why it's so important going forwards in terms of capital.

**Ejaaz** (1:05)
Yeah, and this may feel a little bit different because we're always talking about whose model is the smartest. That's the common conversation. But once you strip away all that abstraction, a token is really just what is output. And that is just a series of electrons flowing through chips, light flowing through fiber, and then heat being pulled out of the system. And all of that requires a lot of energy and electricity.
The money cycle that we're going to talk about has kind of flowed towards this direction as well. I mean, everyone started with the crowd of GPU trade, then it flowed to the semis that exist around them, then it flowed to the memory trade. And now it's kind of moving over to the seemingly the end game bottleneck, which is energy. I mean, US data power center demand, I think is projected to roughly double 80 gigawatts in 2026 to 150 by 2028 So the grid is not really built for that. And there's no shortage of demand for that. And when we think about energy as an idea, even if you believe that the AI bubble is towards the latter end of it, energy is still something that's not going away. We had energy problems prior to the LLM becoming a big deal, from the ChatGPT moment. This is just an extension and an exaggeration on top of that. And it creates a lot of interesting opportunities in the marketplace to actually participate in this bottleneck that is now known as, I guess we call it the energy bottleneck.

**Josh** (2:23)
Yeah, and I think a lot of the reason why people don't talk about this is because it's sort of unsexy, you know, where not everyone is an electrical grid expert, not everyone knows how to network wise between these complex GPUs. I'm still trying to wrap my head around the GPU itself, right? So like the fact that like we're bringing in all these other things, it's quite complex. Now, if you were to look at a budget of spending, right, for an AI lab or someone that's setting up a data center, only 10% of that budget gets allocated to the power and infrastructure side of things. The irony of it now is it's going close to 100% of the bottleneck of the entire thing. So even though it's not the majority of the cost, still 90% of the cost goes into the actual GPUs itself. Those are the most expensive stuff, the memory and the silicon required to build those GPUs.
Only 10% is allocated to power, but that is like the main bottleneck that we're facing today. Now, the lead time, if you are an AI lab, you could be the richest, most important person on Earth right now. If you want to build a data center in the US, the lead time to get the necessary power to your GPUs is five years. There are different sectors within the power thing. It's not just the energy infrastructure grid you need to get access to. You need to get lead times for wiring and to get transformers designed, custom-made, and delivered to you. All of that takes a lot of time and we're looking at basically half a decade. What we've seen a lot of these AI labs start to do now is figure out what alternatives they could potentially procure to help them get GPUs online. Now, you've all heard the news of Meta, Amazon, Microsoft, and Google spending trillions of dollars this year alone. I think it was something along the lines of, actually, it might have hit a trillion dollars for this year, because I think after the last quarterly earnings, they upped their budget. They haven't set those data centers up. In fact, there have been delays. Stargate from OpenAI, which was their biggest project to get compute online, has been delayed multiple times. The point around this is because of compute, and not many people are talking about this yet. I think on today's episode, as we walk down this infrastructure stack, and we did a previous episode before, where we covered all the layers of the AI infrastructure stack, we talked about going from layer 0, which is the Model Labs, down to layer 1, which was the hyperscalers, GPU, semiconductors. We didn't dig as much into layer 5, which is the power and infrastructure side of things. And that's what we're going to do today, and we're going to do it through the lens of full specific companies, starting with Bloom Energy.