Building AI Systems You Can Trust

**Scott Clark** (0:00)
After helping people optimize models for about a decade and a half, I came to a really interesting realization, that it was solving the wrong problem. And foundationally, it was that the thing that's holding back people getting value from these AI systems is not performance. It's not about squeezing out that last half a percent from some eval function or some performance metric. It's about being able to confidently trust these systems. And I can't tell you how many times over the decades we would help someone optimize a system and they would say, okay, well, what did you break? What bad behaviors are you introducing? What lack of robustness do I now have because I've overfit this system? And we're seeing people do the exact same thing again today with LLMs where they're focusing on these high level metrics, these end outputs, these performance evals. And that ends up masking all of these potentially undesired behaviors within the system itself.

**Derrick Harris** (0:56)
Welcome to the a16z AI Podcast. I'm Derrick Harris and I'm joined this week by a16z partner, Matt Bornstein and Distributional co-founder and CEO, Scott Clark for a deep dive on deploying and testing AI systems, particularly but not exclusively LLMs in enterprise environments. We cover a number of topics under this broad umbrella, but the focus was really on how enterprise users can and are trying to establish a trusting relationship with AI so they can actually put it to work on important work and scale their deployments beyond small projects. Genitive AI can be inherently chaotic at a systems level, so it's a process that involves invoking some level of control over usage and lots of monitoring and testing to track behaviors and ensure small changes to one part of the system. The system prompt, for example, don't have unexpected impacts in other parts, including customer-facing outputs. The discussion begins with Scott giving his somewhat tongue-in-cheek definitions of machine learning and artificial intelligence before sharing some of his background, including the hard-earned realization that trust, not performance optimization, is the biggest factor in how heavily large companies will deploy AI. As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Matt Bornstein** (2:27)
So can you just define for us what's machine learning, what's AI? Like as someone who's lived through the ups and downs of this market.

**Scott Clark** (2:34)
I remember answering a similar question, I think on an a16z podcast maybe eight years ago.

**Matt Bornstein** (2:39)
All right, maybe we can pop in the old answer right here.

**Scott Clark** (2:42)
I think the answer is somewhat the same, like machine learning is the stuff that's now become easy, and then AI is all the fun new stuff. And then as soon as it stops becoming the cutting edge, then it just becomes, oh, that's just machine learning.

**Matt Bornstein** (2:54)
It's not magic anymore?

**Scott Clark** (2:55)
It's not magic anymore. And I mean, you can go all the way back to the old like Dartmouth Conference and things like that of like, oh, spell check is AI because that's an intractable problem. But I don't think anybody would constitute that as AI today. Now, of course, with generative AI, I think one of the big differences is, instead of these systems being focused on just kind of like classification or regression, trying to determine whether something exists in a set or what the next element should be, now they're actually much more interactive. And like the types of applications, the types of ways that you can use these systems is becoming so much larger because, I mean, gen is in the gen, the generative aspect of them is in the name, but that's fundamentally different, I think. And I think that's opened this whole new wave of value for enterprises.

**Derrick Harris** (3:42)
Before we get too deep into the state of the world today, can you quick talk through like your background and then how you ended up with a realization that inspired Distributional?

**Scott Clark** (3:50)
It starts about 10 or 15 years ago with my first startup, SIGopt. That company was based off of my PhD research, and it was all about how we help companies optimize AI systems. So how do you get a little bit more performance out of a very sophisticated model?

**Matt Bornstein** (4:06)
And so this is in the sort of traditional machine learning days.

**Scott Clark** (4:09)
Exactly. This was all traditional first wave ML and AI. The definition of AI has obviously shifted over the last decade or so. But it was tuning XGBoost models, tuning like ConvNets, tuning reinforcement learning algorithms, that sort of thing. But it was all about how do I tune all the different parameters. And now, of course, these parameters still exist with temperature and different ways that you can segment various aspects of these foundational models.