Intel is revealing specs for its new GPU D-series AI inference chip at industry summit - Quartz

Skyler Monroe (0:10)

Hey everyone, welcome back to The AI Hardware Show. I'm your host Skyler Monroe, and if you're new here, this is the place where we dig into the silicon, the chips, the data centers, and all the hardware making the AI revolution actually happen. We've got a packed episode today, so buckle up. Before we dive in, huge thanks to our sponsors, Ada, helping businesses integrate AI into their workflows. Ago Consulting, pronounced Ago, your go-to for silicon development consulting from AI to so, and Zen Semiconductor, the umbrella company building the processors and AI fabric behind modern data centers, with ventures like their Sierra RISC vCPUs and Loom AI fabric, plus Ada for deploying AI at scale in any business. Today we're talking Intel's new GPU D270 inference chip, with a wild amount of memory, what Nvidia's Groq 3 LPX means for the Vera Rubin platform, Meta going all in on an expanded Nvidia deal, and a fascinating look at how memory strategies across the industry are starting to diverge. Let's get into it, all right? First up, Intel. So Intel just pulled back the curtain on the GPU D270 at an industry summit, and honestly, there's a lot to unpack here. This is Intel's dedicated AI inference chip, and it's coming in with a headline spec that's hard to ignore, 480 gigabytes of memory. That is a lot of memory, like a whole lot. Now let me give you some context on why memory matters so much for inference specifically. When you're running an AI model, especially one of these large language models, the model's weights, all those billions of parameters, they have to live somewhere while the chip is doing its work. And if your memory can't hold the whole model, you've got a problem. You start getting bottlenecks, you start having to shuffle data back and forth, and your inference speed tanks. So having 480 gigabytes on an inference chip is Intel basically saying, hey, we can fit some really big models on this thing without breaking a sweat. To put that in perspective, think of it like RAM in your laptop. If you've got 8 gigs of RAM and you're trying to run Chrome with 40 tabs, a video editor and a game at the same time, it's going to struggle. Now imagine you've got 480 gigs, suddenly everything fits, everything runs smoothly. That's the vibe Intel is going for here. The chip is called the GPU D270, and it's part of Intel's D-series lineup, which is specifically targeted at inference workloads. And this is a smart move. The AI inference market is exploding right now. Training is still dominated by a few big players. We all know who, but inference, actually running these models to serve users in real time. That market is massive and it's growing fast. Every time you ask a chatbot a question, every time an app does something smart in the background, that's inference happening. So Intel is planting a flag in a very, very lucrative space. Now Intel has had a rough few years competing in the AI chip space. They've been the underdog to Nvidia in a pretty significant way, but moves like this show they're not giving up.

The D270 is a serious product with serious specs, and the fact that they're presenting it at an industry summit means they want the enterprise world to pay attention. They want the hyperscalers to take meetings. They want to be in the conversation. The big question is going to be software and ecosystem. That's always been Intel's challenge in the GPU space. Getting developers and enterprises to optimize their workloads for Intel hardware, when Nvidia's CUDA ecosystem is so deeply entrenched. But for inference specifically, there's a little more flexibility. A lot of inference frameworks are more hardware agnostic than training pipelines. So Intel has a better shot here than they might in the training chip market. And 480 gigabytes of memory. I just want to say that one more time, because it's kind of bonkers, that puts the D270 in a really interesting competitive position, especially for running large models at scale without needing to do a lot of model sharding across multiple chips. If Intel can back that up with solid throughput numbers and competitive power efficiency, this could be a real contender. I'm genuinely curious to see benchmark data when it drops. Keep an eye on this one. Okay, moving on to story number 2, and this one comes straight from the Nvidia developer blog, so we're getting the official word here. Nvidia has detailed how the Groq LPX inference accelerator works within the VeraRubin platform.

And yes, I know Groq and Nvidia in the same sentence. Let's talk about what's actually going on here. So first, a little clarification. The Groq in this context refers to Groq's LPX technology being integrated into Nvidia's ecosystem, specifically the VeraRubin NVL72 platform. This is about how these high-speed inference accelerators plug in to what Nvidia is building with VeraRubin, which is their next-generation AI computing platform. At the heart of VeraRubin NVL72 is a rack-scale system designed for absolutely massive AI workloads. Now the term ultrafast interactivity at long context, let me break that down because it's kind of the crux of why this matters. Long context means the model can handle very long conversations, very long documents, very long inputs without losing track of what was said earlier. If you've ever had a chatbot forget what you were talking about three messages ago, that's a context-length problem. Long context is hard because it requires a ton of memory bandwidth and compute. It's computationally expensive. The ultrafast interactivity part means doing all of that long context processing in real time, fast enough that a human doesn't feel like they're waiting.