Google’s AI Shakeup, Nvidia Weighs Putting Less Memory in Rubin Chips, Meta’s New AI Coding Tools

**Stephanie Palazzolo** (0:13)
Welcome, everyone, to The Information's TITV. My name is Stephanie Palazzolo, and it's Thursday, August 6th. Today on the show, The Information has exclusive reporting on Nvidia's proposed solution to the memory crunch, specifically to incorporate MS memory into its new Rubin chips.
We'll then take a look into how Meta's new coding tools aim to compete with Anthropix Cloud Code and OpenAI's Codex. We'll also unpack the AI leadership shakeup at Google and what it could mean for the future of the company. And to close out the show, we'll talk with the venture capitalists investing in the new startup founded by Google's former chief scientist, Jeff Dean. It's gonna be a great show, so let's get right on into it.
NVIDIA is weighing a radical step to deal with a shortage of advanced high-bandwidth memory chips. Our colleagues Phoebe Liu and Chener Liu broke the story. Phoebe is here to join us now. Welcome to the show.

**Phoebe Liu** (1:08)
Thanks for having me.

**Stephanie Palazzolo** (1:10)
So, Phoebe, how exactly is NVIDIA dealing with the memory crunch?

**Phoebe Liu** (1:15)
Yeah, so our reporting found that NVIDIA is testing versions of its next-generation GPU, so Rubin Ultra, with less memory than it initially planned. So I think in early 2025, Jensen Huang said that Rubin Ultra would have one terabyte of memory per chip, which is a lot, like a lot more than any chip it's ever produced, any chip anyone has ever produced.
The memory crunch has gotten pretty intense since then, and partially because of supply chain pressures, Nvidia is buying so much advanced high bandwidth memory that it's just hard for the suppliers to make enough of it, partially because of that, partially potentially because customers have just as much use for chips with a little bit less memory. Nvidia is testing versions with 256 gigabytes of memory and even 192 with some customers, although it's important to know that nothing has been finalized yet. They're just testing for now.

**Stephanie Palazzolo** (2:21)
It's funny, as you mentioned, that one of the big reasons why we're in this situation to begin with is because over the last year Nvidia's GPUs have become so popular that in some ways, it's led to this memory crunch. It's a bit of a catch-22, it seems. Yeah, I think so.

**Phoebe Liu** (2:37)
Nvidia actually has a team that looks at these supply chain and all the way down to the raw materials issues years ahead, working with kind of projections the company has for how much of each material will need because it is one of the world's largest companies and needs so much of this that it sometimes literally wants more than the entire world can produce. So this is something they think about quite a bit, but even with all the planning they can do, sometimes they can't control every single step of the process, even though I'm sure they would like to.

**Stephanie Palazzolo** (3:14)
And how does having less memory actually affect this new chip's performance then?

**Phoebe Liu** (3:20)
Yeah, so I think because today's most advanced AI models are so big, they have trillions of parameters, that takes a lot of memory to both train and run. So if one specific trip has less high bandwidth memory, to run a model with the same level of performance if all other factors were held equal, you would need more chips, which honestly is kind of good for Nvidia, because they could also in turn sell more GPUs.

**Stephanie Palazzolo** (3:51)
But I'm sure customers aren't thrilled about that idea, right? It kind of feels like, I don't know, do they feel like the rug has kind of been pulled out from under them a little bit and they're like, oh, now I need to buy so many more chips than I originally thought.

**Phoebe Liu** (4:03)
Yeah, so obviously I haven't talked to every customer, but the ones that I did talk to, honestly, I don't think it will affect demand. They're going to want the chips anyway, and hopefully if all things shake out as I'm expecting it to, if chips have less memory, they will also be cheaper. I don't know if that will be kind of a one-to-one cost decrease, but that would be good for customers, especially like the big clouds that are getting more and more worried about how much they're spending on Nvidia GPUs and AI in general.

**Stephanie Palazzolo** (4:39)
So it sounds like even if they have to buy more chips, hopefully they can get a bit of a discount on them.

**Phoebe Liu** (4:44)
I think so, although also important to know that Nvidia hasn't set pricing for Rubin Ultra yet.