As AI chip production ramps up, data centers face power shortages: Semiconductor shipment forecasts are under test - 富途牛牛

Skyler Monroe (0:10)

Hey, everyone, welcome back to the AI Hardware Show. I'm your host Skyler Monroe. And if you're into chips, silicon and the hardware powering the AI revolution, you are in exactly the right place. Big thank you to today's sponsors, Ada, helping businesses integrate AI into their workflows seamlessly. Ago Consulting, that's Ago, your go-to for silicon development from AI to sucks, and Zen Semiconductor building the processors and AI fabric behind modern data centers, with incredible ventures like their Sierra RISC, VCPUs and Loom AI fabric, plus Ada for deploying AI at scale in any business. Today, we've got a packed episode, Power Shortages Threatening the Data Center Buildout, a fascinating new chip interconnect technology backed by Nvidia, open-source AI chip designs for autonomous driving, a wild week in the semiconductor industry, and what Samsung and TSMC's signals mean for AI chip stocks. Let's get into it. All right, let's kick things off with a story that honestly keeps a lot of data center operators up at night. And it's not about chips themselves. It's about the power to run them. As a iChip production is ramping up at an incredible pace, data centers are running smack into a wall, and that wall is made of kilowatts, or really megawatts, sometimes gigawatts. So here's the situation. Semiconductor shipment forecasts are being stress tested right now because the demand for AI chips, your H100s, your B200s, your custom accelerators, has exploded. And that's great news for the chip industry, but here's the thing nobody talks about enough. Every chip you ship needs a home. And that home, the data center, needs electricity. A lot of it. To put this in perspective, a traditional data center rack might draw somewhere around 10 to 20 kilowatts. An AI compute rack loaded with modern GPUs? We're talking 100 kilowatts, sometimes more. That's not a linear jump. That's an order of magnitude increase in power density. You're essentially trying to fit a small power substation's worth of load into a building that was designed for something much more modest, and the grid just isn't keeping up. Power utilities typically plan infrastructure on decade-long time scales.

You don't just call up your local utility and say, hey, I need an extra 500 megawatts by next quarter. That's not how it works. Permitting transmission line upgrades, new substations, this stuff takes years. So what ends up happening? Data center operators are essentially racing to lock up power capacity before their competitors do.

You're seeing companies sign power purchase agreements years in advance. Some are building their own generation capacity, natural gas peakers, on-site solar, even looking at small modular nuclear reactors. It's wild.

And this creates a really interesting bottleneck for semiconductor shipment forecasts. Think about it this way. You can manufacture all the chips you want, but if there's nowhere to plug them in at scale, those chips are sitting in warehouses.

The supply chain isn't just about fabs and packaging anymore. It extends all the way to the electrical grid.

The other angle here is cooling. More power means more heat. Modern AI clusters are pushing the limits of air cooling, which is why liquid cooling is having a major moment right now.

Direct liquid cooling, immersion cooling, these aren't exotic research projects anymore. They're becoming standard infrastructure decisions, and that adds cost and complexity to the build out. What this really signals to me is that the AI hardware ecosystem is maturing in a way that forces us to think about the full stack, not just the chip, but the power delivery, the thermal management, the facility infrastructure. The bottleneck is shifting. It used to be, can we make enough chips? Now it's increasingly, can we power and cool them? That's a fascinating and important evolution in how we think about AI compute capacity.

Okay, story two, and this one is genuinely exciting to me as someone who loves interconnect technology. Upscale AI, which is backed by Nvidia, has just launched something called Token Fabric, and it's designed to link rival AI chips together. Yes, you heard that right. A company with Nvidia backing is building technology to connect chips that compete with Nvidia. Let's unpack that. So first, what is Token Fabric? At its core, it's an interconnect layer. Think of it as a universal translator for AI accelerators. Different chips from different vendors. They all speak slightly different languages in terms of how they move data around, how they handle memory, how they communicate across a cluster. Token Fabric is trying to create a common fabric that lets these chips talk to each other efficiently. The name Token Fabric is actually pretty descriptive if you know how large language models work.

These models process data in chunks called tokens, words, parts of words, little pieces of information. The fabric part refers to the network that stitches compute nodes together. So you're building a high throughput, low latency network specifically optimized for how AI inference and training workloads actually move data. Now, the Nvidia angle here is fascinating and a little counterintuitive at first glance. Why would Nvidia back something that helps rival chips work better?