How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal

**Claire Vo** (0:00)
And still, in, as I say, the year of our Claude 2026, I still talk to engineers that say AI, on our most complicated things, cannot do a good job.

**Ankur Goyal** (0:08)
I so viscerally disagree with that. There's no staff engineer who is running as many rigorous benchmarks and trying out different algorithms and analyzing ideas manually than someone who's using an agent. Everyone should take a hard look in the mirror and reevaluate how they spend their time. There's a lot of interactions that you have or direction that you're giving or decisions that you're making and I think like many of these things to me fit below the agent line. I think the agent line keeps going up.

**Claire Vo** (0:35)
Why do you think this concept is so important to understand? How can you just demystify it for folks who are a little intimidated by it?

**Ankur Goyal** (0:41)
Now that models are so good at actually writing code, one of the best things that we can do is create really hard evals. If you create the right tests and success criteria for a model, then it can be really creative and it can work on this stuff in the background and actually try to improve a bunch of things.

**Claire Vo** (0:57)
I have a lot of people saying, wow, if I go so far as to turn my own taste or my own skills or my own expertise into a system, I'm functionally just building my own replacement.

**Ankur Goyal** (1:08)
We're able to have David's palette applied to more things. I think the quality bar that we're able to hit is higher because we're able to get more things to that bar.

**Claire Vo** (1:20)
Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, and we're on a mission to help you build better with these new tools. Today I have Ankur Goyal, the CEO of Braintrust.
This is a technical one, so if you're a senior or staff engineer or a VP of engineering or CTO, this is one you're really going to want to pay attention to. We're going to talk about how coding agents can help you bite off really technical architecture and infrastructure work in a way that no other human engineer could before. We're also going to demystify evals for folks and just show you exactly how you can use them to make your AI products better without having to touch a thing. Let's get to it. This episode is brought to you by Guru, the AI layer of truth for your company's knowledge. Here's the problem. Your AI is only as good as the information you feed it. Most companies are getting confident but wrong answers from AI because their underlying knowledge is outdated, incomplete, or just plain incorrect. That information doesn't just slow you down.
It costs you money and puts you at risk. Guru solves this by adding a verification layer between your company's knowledge and AI tools. Instead of just hoping your AI gets it right, Guru automatically scores content for accuracy, flags, outdated information, and ensures your team gets trustworthy answers every time. It works with the tools you already use, so you don't have to change how you work. Thousands of companies trust Guru to keep AI accurate and compliant. Ready to stop playing Russian roulette with your company's knowledge? Visit getguru.com to learn more.
Welcome to How I AI. I'm excited to have you here.

**Ankur Goyal** (3:04)
I'm super excited to be here. Thanks for having me.

**Claire Vo** (3:07)
I'm going to make you laugh, but I recently did an episode about the recent GPT 5.5 model release, and I know you and I use Codex. One of the funniest comments in that post was, Claire, can you do an entire episode about TechDev? We were talking before we got on the recording, you're like, how technical and how nerdy is this audience? I'm like, bring it on. We are going to talk a little bit about how you approach engineering, and then how you use AI to do things like optimize slow queries.
Let's hop in. Tell me about your approach to software engineering in the age of AI.

**Ankur Goyal** (3:47)
I spend a lot of time working on software for doing evals and observability, and that's shaped my own perspective about software engineering.
Now that models are so good at actually writing code, one of the best things that we can do is create really hard evals. I'm not talking about AI evals. I mean things like, why is this query so slow? If you create the right tests and success criteria for a model, then it can be really creative and it can work on stuff in the background and actually try to improve a bunch of things. So one of the things that I've spent a lot of time on right now is making the queries that people run in our product faster. People can just write arbitrary queries.