Data Centers Aren't Killing the Grid. They Could SAVE It.

Jason Calacanis (0:00)

All right, everybody, welcome back to This Week in AI. With us today, Varun Sivaram, Sivaram, got it, is the founder and CEO of Emerald AI. They're building software to help data centers cut their power.

Varun, welcome to the program.

Varun Sivaram (0:16)

Jason, thanks for having me.

Jason Calacanis (0:18)

Yeah, so data centers are hated.

Suddenly, they went from locations begging and giving tax breaks for people to build a data center. They were being courted, and then something changed. Maybe there were too many data centers, maybe a little bit of noise pollution on the margins, maybe a little bit of gas pollution on the margins. And of course, key issue, aesthetics and water on the margins. But key issue is, hey, is this going to affect my energy bill? So here we are, you have a company that's helping people lower their energy usage, which I think would address some of those issues. But the big blocker anyway for datacenters is energy consumption. It's easy to find land.

It's not easy to find construction workers, but you can get that done. But the energy seems to be the high order bit. So tell me a little bit about what you're building at Emerald Conductor. Thanks to our friends at PayPal, the exclusive sponsor of This Week in AI. Pay zero processing fees on your first 100,000 ineligible PayPal payment volume. Find out more at paypal.launch.co.

Varun Sivaram (1:28)

Absolutely, Jason.

Jason Calacanis (1:29)

Well, look, we're with the Emerald Conductor because that's your product.

Varun Sivaram (1:32)

Yeah, that's right. Emerald AI is the company we make the Emerald Conductor software platform. I want to be clear upfront, Jason, that the goal is not for AI to use less energy. Actually, AI is going to use a whole lot more energy.

The goal is to enable us to unlock the power grid for AI to use a whole lot more energy, while at the same time keeping local communities' bills affordable and the electric grid reliable.

And there's only one way that you can achieve all three of these things at the same time. More AI datacenters on the grid, but that act as good citizens, friendly to local communities. And that's for those datacenters to be flexible.

So of course, efficiency is important. You know, the Nvidia Verarubin generation today produces a million times more tokens per watt of input energy than the Kepler generation in 2012 But flexibility is a different concept altogether. Flexibility means that every so often, rarely, when the grid is really stressed out, when it's a hot summer day in Phoenix, Arizona, and everyone's air conditioning is turned on and the grid is facing its peak demand, during those moments, if the datacenter can reduce its power consumption, because it's flexible and controllable using our software, then that datacenter can actually fit on the existing grid that we already have, and that datacenter's revenues that it brings into the power system, help to go to pay for the electric grid we already have, and the upgrades that it will need to serve the local community. So again, the only way you get more datacenters, affordable bills, and reliable grids is for datacenters to be flexible. That's the technology that we do to make datacenters grid assets and not liabilities.

Jason Calacanis (3:12)

And that means large swaths of batteries. And so is what I'm seeing here in this looped video on your website, Emerald AI. Are those giant batteries I'm looking at right here, these big black tanks that you have?

Varun Sivaram (3:25)

To be clear, it's not just about batteries. You're looking at GPUs. But in addition to, of course, batteries are one way to do this, right? So you can have a datacenter. And during that hot summer day in Phoenix, the datacenter has got to reduce its power consumption. And one way you could do that is by having massive banks of batteries, right?

But another way you can do it is you can slow down some of the processing that's happening on site during that rare moment. Or you can even, you can pause jobs, you can slow down GPUs, or you can move those jobs from that location, say Virginia or Phoenix, to a different location, say Chicago, at the speed of light.

And if you orchestrate the compute intelligently, we at Emerald believe that that is one of the key ways that datacenters can be flexible grid assets. And in addition, you can have batteries, for example, that can supplement that flexibility from the computations.

Jason Calacanis (4:17)

So you're more in the software business than the battery business here.

You are looking at it saying, hey, the grid is telling us, the grid intelligently tells these datacenters, hey, we're at peak consumption. There's some sort of input. Is there like a standardization for how that occurs and how it gets communicated to people using electricity?