Production AI Engineering starts with Evals — with Ankur Goyal of Braintrust

**Charlie** (0:00)
Welcome back, listeners. This is Charlie, your AI co-host. The LLM Ops landscape has been one of the most competitive battlegrounds of the four wars of the AI stack that we have been referencing throughout this year.
We started covering this in our HumanLoop episode with Raza Habib last year. And this year, a new player has burst on the scene with BrainTrust, led by today's guest Ankur Goyal. BrainTrust started just over a year ago as an incubation with Elad Gill. And this week, BrainTrust raised a $36 million series A at a $150 million valuation with Martin Casado at andreessen Horowitz. We were honored to do the deepest dive into Ankur's journey running a pre-LLM AI company and getting disrupted by BERT, getting acquired by Figma and choosing to start an AI engineering infrastructure company despite having the perfect resume to start a Vector database. As the infra provider of Choice to Notion, Stripe, Versal and others, Ankur also sees the market share of workloads across many of the top AI product teams. So we could not resist asking questions about the rise of Claude 3 and the surprisingly low adoption of open-source models in AI, which also leads to his hot take on fine-tuning. In Latent Space News, you'll notice that we've cut the Suno intro songs. Many of you love them. Some skip them. So we're dropping them for now. alessio was out of town for this episode, but is running the Decibel AI Pioneer Summit next week. Swix will be in New York City for Columbus Day. So if you are in the Tri-State area, come by the Latent Space Meetup linked in the show notes. Watch out and take care.

**swyx** (1:53)
Ankur Goyal, welcome to Latent Space.

**Ankur Goyal** (1:55)
Thanks for having me.

**swyx** (1:56)
Thanks for coming all the way over to our studio.

**Ankur Goyal** (1:59)
It was a long hike.

**swyx** (2:00)
A long trek. You got T-boned by traffic. You are, you were VP of the first VP event at SingleStore. Then you started Impira. You ran it for six years, got acquired into Figma, where you were at for eight months, and you just celebrated your one year anniversary with BrainTrust.

**Ankur Goyal** (2:19)
I did, yeah.

**swyx** (2:20)
What a journey. I kind of want to go through each in turn because I have a personal relationship with SingleStore just because I have been a follower and fan of databases for a while. HTAP is always a dream of every database guy.

**Ankur Goyal** (2:31)
It's still the dream.

**swyx** (2:32)
When HTAP and SingleStore, I think, is the leading HTAP.

**Ankur Goyal** (2:35)
Yeah.

**swyx** (2:35)
What's that journey like? And then maybe we'll cover the rest later.

**Ankur Goyal** (2:38)
But sounds good.

**swyx** (2:39)
We can start SingleStore first.

**Ankur Goyal** (2:40)
Yeah. Yeah. In college, as an Indian, first-generation Indian kid, I basically had two options. I had already told my parents I wasn't going to be a doctor. They're both doctors, so only two options left. Do a PhD or work at a big company. And after my sophomore year, I worked at Microsoft and it just wasn't for me. I realized that the work I was doing was impactful. Like people, there were millions. I was working on Bing and the distributed compute infrastructure at Bing, which is actually now part of Azure. And there were hundreds of engineers using the infrastructure that we were working on. But the level of intensity was too low. So it felt like you got work-life balance and impact, but very little creativity, very little sort of room to do interesting things. So I was like, okay, let me cross that off the list. The only option left is to do research. I did research the next summer and I kind of realized, again, no one's like working that hard. Maybe the times have changed. But at that point, there's a lot of creativity, and so you're just bouncing around fun ideas and working on stuff and really great work-life balance. But no one would actually use the stuff that we built, and that was not super energizing for me. So I had this existential crisis, and I moved out to San Francisco because I had a friend who was here and crashed on his couch and was talking to him and just very, very confused. He said, you should talk to a recruiter, which felt like really weird advice. I'm not even sure I would give that advice to someone nowadays, but I met this really great guy named John, and he introduced me to 30 different companies. I realized that there's actually a lot of interesting stuff happening in startups, and maybe I could find this company that let me be very creative and work really hard and have a lot of impact, and I don't give a shit about work-life balance. So I talked to all these companies, and I remember I met MemSQL when it was three people, and interviewed, and I thought I just totally failed the interview. But I had never had so much fun in my life. I remember I was at 10th and Harrison, and I stood at the bus station, and I called my parents and said, I'm sorry, I'm dropping out of school. I thought I wouldn't get the offer, but I just realized that if there's something like this company, then this is where I need to be. Luckily, things worked out, and I got an offer and I joined as employee number two, and I worked there for almost six years, and it was an incredible experience. I learned a lot about systems, got to work with amazing customers. There are a lot of things that I took for granted, that I later learned at Impira that I had taken for granted, and the most exciting thing is I got to run the engineering team, which was a great opportunity to learn about tech on a larger stage, recruit a lot of great people, and I think for me personally set me up to do a lot of interesting things after. Yeah.