AI's Impact on Memory: Exploring HBM, DRAM & NAND market trends

SPEAKER_1 (0:00)

Hello, and welcome back to TechInsights, the Chip-Observer. Home to all things semiconductor related, covering industry trends, innovations, and key developments.

So we've been away on hiatus over the summer, but now we're back with an all new episode and subjects to take a look at. Recently, TechInsights hosted a series of memory-related webinars, and the following is a quick recap on some of those subjects. Along in this episode, we'll hear from the likes of Mike Howard, Director of DRAM and Memory Markets, Walt Kuhn, Director of NAND and Memory Markets, and Dr. Jeong Dong-chou, Senior Technology Fellow. And we'd also like to acknowledge Mike Walden, VP of Critical Materials Information, who, without his guidance and input, this episode wouldn't have been possible.

So, on this episode, we're exploring the profound impact of artificial intelligence on the memory industry, covering how the rapid rise of AI and agentic workflows is driving unprecedented demand for HBM, DRAM, and NAND, and how this might affect markets over the coming years. First real question, from Mike Howard. As we move toward more agentic AI, how does that increased demand ripple through the accelerator and memory markets?

Mike Howard (1:41)

As we get more agentic AI, we see the increased value of the AI, which drives more demand, which means we need more accelerators, more HBM, and the cycle goes on and on. So by our estimates, given sort of the, at least the near to midterm projections of where we need to see AI data center gigawatt of capacity installed go, we think this market is going to be relatively undersupplied for at least the next several years.

SPEAKER_1 (2:03)

Mike, looking at the big picture, how is AI demand reshaping the memory market right now?

Mike Howard (2:09)

Well, there's a number here. First of all, it definitely appears that today we continue to be compute constrained. Memory essentially goes one to one with compute. So if we're compute constrained, it means that we are memory constrained. And I think that the big thing that's shaping the dynamics in the industry right now is that the growth in demand from AI has come on so quickly that it completely caught suppliers flat-footed. Suppliers, you'll remember, were in a downturn in 2023, where everybody lost money right around the launch of Chad GPT.

And so they have been struggling just to keep their heads above water with this new rise in demand. And when we look at the AI demand, it really is the demand that's driving the flywheel. So an accelerator shift needs HBM memory, and every accelerator that shifts today, especially in the age of agentic AI, is driving increased CPU demand. Those CPUs then need EDR demand, which is making the market even tighter. But as we get more agentic AI, we see the increased value of the AI, which drives more demand, which means we need more accelerators, more HBM, and the cycle goes on and on. So by our estimates, given sort of at least the near to midterm projections of where we need to see AI data center gigawatt of capacity installed go, we think this market is going to be relatively undersupplied for at least the next several years.

SPEAKER_1 (3:24)

So it's safe to say suppliers were caught completely off guard. What's changed in how the industry operates now?

Mike Howard (3:30)

So the four key shifts that we see today, which really are a departure from the past, are first of all, there's this reset around pricing and long term customer agreements, which is fundamentally raising the margin floor. So the LTAs, they give the suppliers much better visibility into long term demand, and they sort of bracket, at least where the pricing could go long term. Contracts are not new to the memory markets. The difference here is that these contracts have a lot more teeth, there's a lot more money involved, and given the long term probability, high probability of an undersupplied market, customers are much more willing to sign up to more binding terms. So the LTAs and the HBM pricing, they're changing the margin structure within the industry. Now, historically, that wouldn't have been feasible. The industry didn't generate enough money to spend that much. Given today's pricing environment and the rampant demand from AI, there's going to be a vast amount of free cash flow by our estimates by 2031 over $4 trillion in free cash flow from just the DRAMP space alone. So massive expansion in spending and capacity build.

SPEAKER_1 (4:34)

So what's really changed in the market this time around? What are we looking for?

Mike Howard (4:38)

For the last, call it 20 years, we've really been dependent on PC, and then that transitioned into the smartphone demand. Data center has always been there, but it's been a smaller part of the market. And today, it is increasingly really the only care about segment almost in the market. And so, AI continues to improve as it improves, as I mentioned, demand continues to grow. And this shift to agentic is driving, I mean, first of all, let's concede that agentic is something that kind of came out of nowhere, call it six, eight months ago, and it is driving massive amounts of CPU and thereby DDR demand. The truth is actually the exact opposite. This is customers reacting to a given or a finite pool of VRAM availability and trying to maximize their products throughput. So on the side of an accelerator, if you make accelerators and you're constrained on HBM, then you can either put a lot of HBM on a few accelerators, or you can put less HBM on more accelerators. And so this despecking that we're seeing is customers trying to make their bits go further. And for these four sort of themes right now are really what's shaping the market and is what's sort of divergent from the past cycles.