HS143: Why We Don’t Believe in Self-Healing AI (Sponsored)

Johna Till-Johnson (0:03)

Hello, this is Johna Till-Johnson and John Burke from Heavy Strategy, and we're delighted to be here with Rekha Shenoy, CEO of BackBox and Richard Phillips, VP of Product and Engineering at BackBox. They're joining us to discuss what can go wrong when AI operates without enough context or control.

Rekha Shenoy (0:21)

Hi there, happy to be part of this program today.

Richard Phillips (0:26)

Hey Johna, Johna, thanks for having us. Absolutely.

John Burke (0:28)

Always a pleasure.

And I'll kick things off. I think we're all familiar with the promises of AI for fully autonomous, network automation, self-healing, faster problem resolution, lower costs. I know I was hearing promises like this from all people, IBM back in the early 2000s. And these have been promises since way before LLMs. So with LLMs, things supposedly are more plausible than ever before.

Is it? Is it a good thing?

Rekha Shenoy (1:04)

It is a good thing, John, but it's also become very, very real.

Last year alone, we had 49,000 new CDEs. That's a 20 percent growth over 2024 But this year, we're just going to blow that number out because of AI.

You think about project methods, you think about project glasswing, and you just see the pace at which we're finding vulnerabilities on network infrastructure, it's completely unprecedented, and that's new in terms of how AI is just about the automation and self-healing. It's changed the world for our customers. Meanwhile, our vendors, the network vendors are pushing their customers to upgrade and patch at a pace that they've just never done before. Think about the complexity of the environment that today's networks are designed around. It's sheer impossible to keep up with that pace. That's just an example of how AI became very real for our customers in a negative way.

John Burke (2:06)

So it's not so much about making network management easier and potentially cheaper for the organization anymore. Now it's just about survival.

Rekha Shenoy (2:18)

Well, there's still the promise.

Go find yourself a network infrastructure team that hasn't had a business owner come in and say, hey, can we use some AI instead of more people, and find one that hasn't had an eye roll moment like that. So yes, there's still the promise. It can be cheaper and all of that. But to the point of realism, there's also plenty of ways to do that and get it wrong. Because network infrastructure isn't there just to play with AI. It's there to keep very mission-critical businesses and processes up and running. SLAs matter, bandwidth matters, availability matters, performance matters.

In that world, you can't just throw an AI that is not accountable at all into the mix, and just go make it work. So I think there's a deep understanding with the network folk that there's got to be a better way to do it. But in the absence of that, they're still accountable to those SLAs, and AI is not necessarily the solution as a, just throw it in and replace an engineer.

Johna Till-Johnson (3:24)

So Rekha, it sounds almost like you're saying, would it be a good thing? Maybe, maybe not, depending on how well you do it, but it is a necessary thing given the change in the broader context, the emergence of mythos and its peer models, and the challenges that are posed to network engineers and network architects and network operators by the vendors, by AI in the hands of bad guys.

It's possible to try to go into autonomous network automation and do it very badly, but you still have to try because the risk of not doing it is even greater than the risk of doing it wrong, it sounds like.

Rekha Shenoy (4:04)

I think what's missing in this mix, and I think the piece that customers really want to hear is responsible AI.

Nobody is saying AI can be responsible and accountable. That's what they want.

That's the game changer in this mix. What I mean by that is, let's not forget all of the workflow and the accountability and the auditability, and all of those things that we built, processes we've built, and SLAs that we have committed to the business when we do this. How do we build a system that is responsible to all the things we as a network team are accountable for? Now, let's take this mythos problem that we just talked about. In the process of all of the work that these network infrastructure teams are doing, the most advanced teams are probably patching once a quarter, and most likely, they're patching once a year, is more the norm in today's world. What we're saying is, the pace at which hackers are attacking these vulnerabilities went from weeks and months to hours or days. They need a better solution, but the better solution is not throw AI into the mix willy-nilly. The better solution is, can we get responsible AI that will massively reduce time and effort for us? Just getting really real and pragmatic. People are looking for more responsible AI.