AI Chip Startup d-Matrix Taps Nvidia's NVLink Fusion to Break Into Data-Center Inference Market - finance.biggo.com

Skyler Monroe (0:10)

Hey, everyone, welcome back to the AI Hardware Show. I'm Skyler Monroe, your guide through the wild world of silicon semiconductors, and the hardware that's actually making AI happen.

Big thanks to our sponsors today, Ada, helping businesses integrate AI into their workflows, Ago Consulting that's, ah, go, your go-to for silicon development from AI to so, and Zen Semiconductor, building the processors and AI fabric behind modern data centers, with ventures like their Sierra RISC-V CPUs and Loom AI fabric, plus Ada for deploying AI at scale in any business. Today we've got a packed show, a scrappy chip startup getting cozy with Nvidia's Interconnect Tech, Groq's new inference accelerator landing on Vera Rubin, TSMC printing record revenue, a deep dive into the substrate bottleneck nobody's talking about enough, and Samsung planting a flag in Japan for advanced packaging. Let's get into it. All right, kicking things off, there's a really interesting story about D-Matrix, an AI chip startup that's just announced it's tapping into Nvidia's NVLink Fusion to break into the data center inference market. And if you've been following the chip space at all, you know this is a pretty bold move. So let me give you a little context first. D-Matrix has been building what they call a digital in-memory compute architecture. The idea being that instead of constantly shuttling data back and forth between memory and a processor, you do the computation right where the data lives.

That's a fundamentally different approach compared to traditional GPU style inference. And for certain workloads, especially inference-heavy ones, it can be dramatically more efficient.

Now here's where the NVLink Fusion piece gets really interesting. NVLink has traditionally been Nvidia's proprietary high-speed interconnect. It's what lets multiple GPUs in a system talk to each other at insane bandwidth. Think of it like a super highway between chips versus the slower surface roads of something like PCIe. NVLink Fusion is essentially Nvidia opening up that super highway, or at least a lane of it, to third-party silicon. What that means for D-Matrix is huge. Instead of trying to compete head-to-head with an H100 or a Blackwell chip on raw training performance, which, let's be honest, would be a losing battle, they can now plug their inference-optimized hardware directly into an Nvidia-centric data center infrastructure. Their chip becomes a compliment, not a competitor.

Think of it like this. Nvidia is the city's main power grid, and NVLink Fusion is Nvidia handing out approved adapters so other appliances can plug in. D-Matrix gets to run on the grid, customers get to use best-of-breed inference silicon, and Nvidia keeps its ecosystem sticky. Everyone kind of wins. From a market strategy standpoint, this is smart for D-Matrix. Inference is where the volume is going to be, training gets all the headlines, but once a model is built, you're running inference billions of times a day.

The economics of inference matter enormously. Cost per token, latency, power efficiency, these are the metrics hyperscalers obsess over. And D-Matrix has been laser-focused on that problem. Their Corsair architecture was already designed with inference throughput and memory efficiency in mind. Adding NVLink Fusion compatibility means they can slot into existing Nvidia-heavy racks without a massive infrastructure overhaul on the customer side. Now, there are real questions here, too.

NVLink Fusion is still relatively new, and the ecosystem around it is just being built out. How seamless is the actual integration? What does the software stack look like? Because hardware is only half the battle if the programming model is painful? Adoption stalls no matter how good the silicon is. But directionally, this move tells you something important about where the industry is heading. We're moving away from a world where one ship does everything toward a more heterogeneous compute model, different accelerators, optimized for different tasks, all connected by high bandwidth interconnects. NVLink Fusion is Nvidia's bet that they can be the connective tissue of that future, even if they don't own every piece of silicon in the rack. For investors and engineers watching this space, D-Matrix is definitely a company to keep your eye on. They've been heads down for a while, and this partnership signals they're ready to actually compete for real data center dollars. That's a significant inflection point. Okay, story two, and this one comes straight from Nvidia's developer blog, so we're getting some meaty technical detail here. Nvidia has published a deep dive on how Groq's LPX inference accelerator works within the Vera-Rubin platform, specifically around long-context, ultra-fast interactivity. And there's a lot to unpack here. First, let's set the stage. Vera-Rubin is Nvidia's next-generation platform, named after the astronomer Vera Rubin, keeping with their tradition of naming architectures after scientists. The flagship configuration is the NVL-72, which is a massive rack-scale system with 72 Rubin GPS, connected via NVLink. That is a beast of a machine. Now, Groq, not to be confused with Groq, Elon Musk's chatbot, Groq is a company that's built inference accelerators called Language Processing Units, or LPUs. Their whole thesis is deterministic, ultra-low latency inference. No caching tricks, no speculative execution weirdness, just blazing fast, predictable token generation.