SK hynix, In Collaboration With SanDisk, Unveils The New High Bandwidth Flash (HBF) Standard, Helping To Resolve AI Inference Bottlenecks, Targeting Up To 3TB/s Bandwidth

**Skyler Monroe** (0:10)
Hey, everyone, welcome back to The AI Hardware Show. I'm your host Skyler Monroe, and if you're into the silicon and systems powering the AI revolution, you are in exactly the right place. Huge thanks to our sponsors today. Ada, which helps businesses actually integrate AI into their workflows in a meaningful way.
Ago Consulting, that's Ago, your go-to for silicon development consulting from AI chips all the way to full sock design and Zen Semiconductor, an umbrella company building the processors and AI fabric behind modern data centers. We're talking their Sierra RISC-V CPUs, their Loom AI fabric, and Ada for deploying AI at scale. These folks are doing incredible work, and I'm glad to have them along for the ride. On today's show, we've got a fascinating new memory standard that could reshape how AI inference works. A custom silicon startup that just landed a major strategic investment. And we're breaking down the chip stock debate. Everyone seems to be having PlusMeta's latest hardware moves. Let's get into it.
All right. Kicking things off with a story that I think is genuinely underappreciated right now.
SK Hynix and SanDisk have teamed up to introduce something called High Bandwidth Flash, or HBF. And it's positioning itself right between two worlds that have historically been very far apart. So let me set the scene here. When you're running AI inference workloads, meaning you're actually deploying a trained model to answer questions, generate content, whatever it might be, you've got this really awkward performance cliff inside your system. On one end, you've got HBM high bandwidth memory, which is that super fast stacked dram that sits right on your AI accelerator. It's blazing fast. We're talking bandwidth in the terabytes per second range, but it's also expensive and you can only fit so much of it. On the other end, you've got enterprise SSDs, which give you massive storage capacity. But the bandwidth is just nowhere near what HBM offers. It's like comparing a fire hose to a garden hose. So what happens? Your AI chip is sitting there, screaming for data and the SSD just can't feed it fast enough.
This is a real practical bottleneck that affects inference performance in production environments every single day. That's where HBF comes in. The idea is to fill that middle gap, something with more capacity than HBM, but dramatically more bandwidth than a traditional enterprise SSD.
HBF uses NAND flash dyes, the same fundamental technology inside your SSDs, but it stacks them vertically, either in 8 or 16 layer configurations, and connects them using a wide high bandwidth interface, similar in spirit to how HBM is architected. Think of it like this. Instead of sending data down a single lane highway, like a traditional SSD, HBF opens up a multi-lane expressway. You're moving a lot more data in parallel at the same time. The specs here are genuinely impressive. A single HBF configuration can hold up to 512 gigabytes of flash storage, and the bandwidth target is up to 3 terabytes per second. That's not quite HBM territory, but it is absolutely in a different universe compared to any enterprise SSD on the market today. And because it's man-based, you get that density advantage. You can pack a lot more data near the processor than you ever could with HBM alone.
Now, why does this matter specifically for inference? Great question.
When you're running large language models at scale, the model weights, the actual numerical parameters that define the model's behavior are enormous. We're talking hundreds of gigabytes, sometimes over a terabyte for the biggest models. You can't always fit all of that in HBM, so systems end up doing a lot of data shuffling between fast memory and slower storage. And that shuffling kills your throughput and drives up your latency. HBF could sit in that hierarchy as a high-speed staging layer. You keep your hot frequently accessed weights in HBM and your warm or less frequent data in HBF, and you access it fast enough that the system barely notices the difference. What's also notable here is the ecosystem angle. SK Hynix and SanDisk aren't just building a product, they're introducing a standard. They want multiple companies to build HBF compatible products, which means if this takes off, you could have a whole ecosystem of vendors offering compatible solutions. That's a smart move strategically.
Rather than locking customers into a single supplier, you're building a platform, and platforms tend to be stickier and more broadly adopted. Now, there are still open questions. How does HBF interface with existing AI accelerator architectures? Is it a drop-in addition, or does it require redesigning how the memory hierarchy is organized? Those are engineering challenges that the industry will need to work through, but the fundamental problem being solved here is real and growing. As AI models get bigger and inference workloads scale up, the gap between compute performance and storage bandwidth becomes more and more painful.