HS147: Architecting Infrastructure to Meet Your AI Ambitions (Sponsored)

Jon Attil-Johnson (0:03)

Hi, I'm Jon Attil-Johnson, CEO of Numertis, here with my co-host, Jon Burke, CEO of Numertis. And you're listening to Heavy Strategy, the show that tries to ask the right questions, not give the right answers. Today, we're joined by Sabina Anya, Chief Technologist for the VMware Cloud Foundation, VCF at Broadcom. And on today's show, we're going to address the challenge of architecting infrastructure for AI.

Sabina, what can you tell us about the challenge?

Sabina Anya (0:31)

Hello there.

Let's start a little bit at the beginning. There's any number of myths and ideas about the challenge of AI. Some folks go to, which model? What are we going to use?

How many parameters can we throw there? But the main challenge isn't bringing that new piece, it's literally everything underneath it, because you need to put something behind this model, the engine that will deliver everything. So, most companies have figured out some degree of a use case, or for the love of God, I hope you did, and what your budget behind this is. And it's usually very disproportionate nowadays, where a lot of things, like this AI is supposed to solve a lot of problems, you know, go-do kind of thing. Now, let's go back to infrastructure. What you usually don't have is the infrastructure that can take this to production within, let's say, a quarter without a million change requests, right? AI is one of the first workloads that makes things like tech debt incredibly visible very quickly.

Jon Burke (1:33)

All the bad choices you swept under the rug or pushed into the closet, they're now popped out and made visible.

Sabina Anya (1:40)

Absolutely. And if you've ever remodeled a house or had that whole thing where you end up having children, you know that it's the kind of challenge that you want to design beyond just the extra room, right? If you're setting a second floor to your house, if you manage to go through all of that, you need to rethink your wiring, your electricity, your water, all these things. It's not just like, oh yeah, we'll just build above, it's fine. Or even better, we're having a kid, we're going to put it in the kid's room, and we're going to make sure it just stays there. Anybody that's had a kid will tell you there's no way, maybe like the first six months when it sleeps so much, maybe, hopefully. But after that, it's just basically this, it's everywhere. It shows you all of those doors that don't close well, that everything is essentially a death trap.

Jon Attil-Johnson (2:25)

All the plugs you can stick your fingers into.

Sabina Anya (2:28)

Yes, yes. Being a child that has this death theory, I can tell you kids get very creative.

Jon Attil-Johnson (2:37)

So you've laid out the challenge. In your experience, do senior leaders actually understand this challenge, or is it something that the infrastructure folks are going to have to articulate, like one of the hard questions to ask?

Sabina Anya (2:54)

Now, the main challenge with AI, especially from a strategic perspective, is that it is a two-world, two-story kind of problem, right? You have this challenge in the leadership world, where every C level and V level has been tasked with the painful thing of finding, how is AI going to bring us efficiency, productivity, and insert your favorite buzzword of the week, right? At the same time, you as the CISO, the whatever, infrastructure manager, security responsible, you have to figure out, what's my risk? Am I doing this right? What are all the moving pieces behind this? Literally, the entire board is affected by the AI challenge, but their view on average, and I guess the larger the company, the more this is true, is about risk, revenue, ROI, TCO, that part, right? And you expect your delivery teams to just get it done. But now you have this divide between leadership and infrastructure, and especially if you go all the way to the system engineer going like, you want me to do what with what now?

Which is a very honest reaction of, we're still trying to finish last year's security project that meant getting rid of our any-any rule from the firewall, and now you're coming to me with frontier AI threats.

Jon Burke (4:11)

It's a two-edged sword. It's cutting you in both directions. How is this strategic push, supposedly strategic push to deploy AI, how is that going to hold you back from accomplishing your previous deferred goals and dealing with your existing technical debt?

And at the same time, how is the infrastructure that you have going to become an impediment to realizing all those AI strategies and accomplishing those goals? It's like you're looking at this thing from either edge, and from either edge, it's cutting you.