From Kubernetes to AI maximalism | Stacklok's Craig McLuckie

**Ben Lloyd Pearson** (0:06)
Welcome to Dev Interrupted. I'm your host, Ben Lloyd Pearson.

**Andrew Zigler** (0:10)
And I'm your host, Andrew Zigler.

**Ben Lloyd Pearson** (0:12)
This week, I'm sitting down with Craig McLuckie, CEO of Stacklok and co-creator of Kubernetes. Andrew, I'm gonna admit I was a little starstruck when I met Craig because my career really took off when Kubernetes was becoming big, so super excited to get to interview him. And he joins me to advocate for an AI maximalist philosophy that is shifting the leadership mindset from asking if we can use AI to demanding to know why we can't. But first, let's discuss some of the stories that caught our eyes this week. So today's stories, we have how to make probabilistic AI more deterministic, LLMs and Dunning-Kruger, replacing glue people with workflows, research about agentic refactoring, and a fun article about dishwashers to wrap it all up. So why don't we just start at the top, Andrew? In this article, making probabilistic systems deterministic comes from someone I actually had met in real life, Josh Phillips. Met him recently at an event, just the morning after he had taken this vibe coding workshop. Josh is this self-described manager of managers who had been pretty disconnected from the coding world for quite a long time now. He felt like he was missing out on the most exciting time of his career to be a developer with all this hype around AI and everything. He had just an incredible amount of energy around this idea of getting back into writing software with AI. This article is yet another story I've been following from him over the last few months. He covers how AI models, they are inherently probabilistic. They give different answers each time, even if you give them the same question, which that type of thing is, that behavior is unacceptable if you're in a high stakes field. Something like healthcare or clinical research, for example.
And Josh built this proof of concept system to effectively make AI more deterministic, reproducible and auditable. And he did that by applying some of the same engineering rigor that they use in finance and high compliance industries. So he was making the argument that you can't get determinism just by setting an LLM's temperature to zero. You also have to capture the entire retrieval context, for example, so that you can reproduce reasoning. Josh focuses on measurable quality metrics, things like groundedness, answer relevance, context relevance, and he's distinctly arguing against a vibe-based judgment on AI. You should make this quantifiable as much as possible. So in this example that he built, it has things like every AI generated answer having verifiable citations to real clinical trial IDs, which is designed to help reduce hallucinations and make it more auditable for regulators and researchers. He also outlines how he treats AI pipelines sort of like a distributed system. So you need to have full observability with logs, metrics, traces, audit trails. All of that is required to diagnose and recreate any answer. But what he was trying to accomplish with this is proving that LLM systems can be deterministic and trustworthy, especially when you're using really strong guardrails. And it's a really great just practical approach to enforcing more deterministic behavior from LLM. And Andrew, I know you have a lot to say about tracing and auditing when it comes to AI agents. So what do you think about this?

**Andrew Zigler** (3:33)
First off, I loved this article, but Josh, it really hits on a lot of the points that I've been experiencing in building and working with agentic systems and especially making these pipelines, right? Taking the probabilism and dialing it down to be something more deterministic, like cutting away the things that we often think of as the features of AI to make it more reproducible and understandable. That's the heart of what I've been really focusing on and how I've been using and building AI here at Dev Interrupted and LinearB, especially making pipelines that help our marketing team, for example. And it's really interesting to bridge into this go-to-market engineering world with the help of AI. And all of that has been made possible by the power of evals and observation and auditability of the process that you're building. Because if you're not using those evals and those tracing methods and you don't understand how what going in influences what comes out, then you're not really building agentic systems. You're just feeding an expensive parrot in hope that it says that you want it to say. And you have to really instrument it to that reason. So really, really resonated with everything from Josh here. He talks about how dialing down that temperature to zero isn't all you got to do. You have to capture all of that context. This is exactly what I talked about when building our AI code review harness for testing all of the different AI code review tools. We recently did a benchmark on that as part of achieving that. The system needed to capture all of the context of every kind of bug and every kind of tool that was thrown at it so that they could be reproduced and understood later down the line. So definitely check out this article. Also check out Dev Interrupted's article last week.