HN836: Optimize AI Traffic on the WAN with Ciena’s Integrated IP Networking (Sponsored)

Drew Conry-Murray (0:00)

Welcome to Heavy Networking, the podcast for listeners with strong opinions on EVPN VXLAN versus TradCore. I'm Drew Connery-Murray, Ethan Banks is away on a walkabout. Today, I'm talking optics and more with sponsor Ciena. And if you're guessing this is an episode about optics and AI data centers, not quite. Today's focus is the WAN infrastructure that carries traffic among users, businesses, and the data centers running AI inferencing workloads. We're going to delve into the evolution of IP networks and coherent optical technologies that enable the movement of AI data across the Internet. As more applications and services are relying on AI inferencing, that means more traffic on the WAN, and while more bandwidth can solve a lot of problems, bandwidth is not unlimited, and AI inference traffic still has to share pipes with other flows. That means things like latency, determinism, security, and traffic engineering are just as important as the size of the pipe. So Ciena is here to talk about what it's doing with routing, switching, and coherent optics technologies to help engineers steer traffic across the WAN as effectively and efficiently as possible. My Ciena guests are Rafael Francis, Senior Director of Product Line Management, and Vinny Santos, Director for Routing and Switching Portfolio Marketing.

Rafael and Vinny, welcome to Heavy Network. And Rafael, I'll ask you to start by explaining the problem that AI inferencing presents for the WAN.

What's different about AI inferencing traffic that requires this kind of special attention?

Rafael Francis (1:19)

Yeah, thanks, Drew. And I just want to say it's good to be here. It's my first time on the podcast, but I've certainly enjoyed and appreciated your content over the years.

Drew Conry-Murray (1:31)

Thank you.

Rafael Francis (1:32)

Good to be here. So before I get into the details of inference, just to set a little bit of context. So Ciena, as a company, we offer solutions to service providers, hyperscalers, neoscalers, what we call neoscalers, others refer to them as neocloud providers. And so these all play a role in the whole ecosystem behind AI-driven investments that are happening in networks. So we have a front row seat, and we have this distinctive view in that we operate at the intersection of IP routing and coherent optics and software control of all this. They all kind of play a role in what's happening in AI investments. And inference is really where a lot of the focus is now placed.

To this point, there's been a lot of investment in the training of these AI models, the race to build the best models, investing in AI data centers, GPU clusters, in what we call the scale up, scale out, and scale across parts of these networks. But really, it's this next, those we refer to as the AI factories, if you will, the building of these LLMs. But this next wave or phase of investment is really, okay, how do you monetize all that investment? So a lot of money being spent, hundreds of billions of dollars annually, as you know. And so now, how do you monetize that? And that's where the consumption of these models comes into play from enterprises and consumers. And that's what AI inference is all about, right? How do we now interact with these models to gain insights and predictions and answers for things? And obviously, that inference for the end users, consumers, enterprises has to traverse Metro and last mile networks to reach those end users. And that's where the service providers become a big part of that value chain. And so that's where things are now focused. Now, in terms of how AI inference is different, certainly compared to just traditional broadband networks, enterprise, residential, et cetera, consumers, certainly bandwidth is going to be increased, right?

Especially with multimodal AI, more video-rich AI and imaging. That's one way it's going to change.

Drew Conry-Murray (4:04)

So not just text-based AI, not just chats. I'm actually asking my AI to build a video for me or give me some images for marketing or work or whatever.

Rafael Francis (4:12)

Exactly. So bandwidth.

Vinny Santos (4:13)

So bandwidth is just like the beginning, right?

Drew Conry-Murray (4:16)

What was that, Benny?

Vinny Santos (4:17)

LLM is just like the beginning of the story. We are like the tip of the iceberg. There's a lot that will come with AI and multimodal inference that will really change the market.

Rafael Francis (4:29)

And a lot of that is, some of that is also more upstream, right? So now I'm providing a lot of context, which could also be multimodal. And so where traditional networks were very downstream-oriented, I need some more upstream capacity perhaps now too, right? Latency is important, obviously, with inference. And network latency can matter more than compute latency, right?