Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon

**Lenny Rachitsky** (0:00)
We worked on a guest post together. We had this really key insight that building AI products is very different from building non-AI products.

**Aishwarya Naresh Reganti** (0:08)
Most people tend to ignore the non-determinism. You don't know how the user might behave with your product, and you also don't know how the LLM might respond to that. The second difference is the agency-controlled trade-off. Every time you hand over decision-making capabilities to agentic systems, you're kind of relinquishing some amount of control on your edge.

**Lenny Rachitsky** (0:25)
This significantly changes the way you should be building product.

**Kiriti Badam** (0:28)
So we recommend building step by step. When you start small, it forces you to think about what is the problem that I'm going to solve. In all these advancements of the AI, one easy slippery slope is to keep thinking about complexities of the solution and forget the problem that you're trying to solve.

**Aishwarya Naresh Reganti** (0:42)
It's not about being the first company to have an agent among your competitors. It's about have you built the right flywheels in place so that you can improve over time.

**Lenny Rachitsky** (0:50)
What kind of ways of working do you see in companies that build AI products successfully?

**Aishwarya Naresh Reganti** (0:55)
I used to work with the CEO of now Rackspace. He would have this block every day in the morning, we should say, catching up with AI 4 to 6 a.m. Leaders have to get back to being hands-on. You must be comfortable with the fact that your intuitions might not be right. And you probably are the dumbest person in the room and you want to learn from everyone.

**Lenny Rachitsky** (1:13)
What do you think the next year of AI is going to look like?

**Kiriti Badam** (1:16)
Persistence is extremely valuable. Successful companies right now building in any new area, they are going through the pain of learning this, implementing this and understanding what works and what doesn't work. Pain is the new moret.

**Lenny Rachitsky** (1:29)
Today my guests are Aishwarya Reganti and Kiriti Badam. Kiriti works on Codex at OpenAI and has spent the last decade building AI and ML infrastructure at Google and at Kumo. Ash was an early AI researcher at Alexa and Microsoft, and has published over 35 research papers. Together, they've led and supported over 50 AI product deployments across companies like Amazon, Databricks, OpenAI, Google, and both startups and large enterprises. Together, they also teach the number one-rated AI course on Maven, where they teach product leaders all of the key lessons they've learned about building successful AI products. The goal of this episode is to save you and your team a lot of pain and suffering and waste of time trying to build your AI product. Whether you are already struggling to make your product work or want to avoid that struggle, this episode is for you. If you enjoyed this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube, it helps tremendously. And if you become an annual subscriber of my newsletter, you get a year free of a ton of incredible products, including a year free of lovable, replet, bold, gamma, N8N, linear, Devon, post-hoc, superhuman, dscript, whisperflow, perplexity, warp, granola, magic pad, androidcast, chapter D, mobid, and stripe atlas. Head on over to lennysnewsletter.com and click product pass. With that, I bring you Aishwarya Naresh Reganti and Kiriti Badam, after a short word from our sponsors. This episode is brought to you by Merge. Product leaders hate building integrations. They're messy, they're slow to build, they're a huge drain on your roadmap. And they're definitely not why you got into product in the first place. Lucky for you, Merge is obsessed with integrations. With a single API, B2B SaaS companies embed Merge into their product and ship 220 plus customer-facing integrations in weeks, not quarters. Think of Merge like Plaid, but for everything B2B SaaS. Companies like Mistral AI, Ramp, and Dorada use Merge to connect their customers as accounting, HR, ticketing, CRM, and file storage systems to power everything from automatic onboarding to AI-ready data pipelines. Even better, Merge now supports the secure deployment of connectors to AI agents with a new product so that you can safely power AI workflows with real customer data. If your product needs customer data from dozens of systems, Merge is the fastest, safest way to get it. Book and attend a meeting at Merge.dev slash Lenny, and they'll send you a $50 Amazon gift card. That's Merge.dev slash Lenny. This episode is brought to you by Strelia, the customer research platform built for the AI era. Here's the truth about user research. It's never been more important or more painful. Teams want to understand why customers do what they do. But recruiting users, running interviews, and analyzing insights takes weeks. By the time the results are in, the moment to act has passed. Strelia changes that. It's the first platform that uses AI to run and analyze in-depth interviews automatically, bringing fast and continuous user research to every team. Strelia's AI moderator asks real follow-up questions, probing deeper when answers are vague, and surfaces patterns across hundreds of conversations all in a few hours, not weeks. Product, design, and research teams at companies like Amazon and Duolingo are already using Strelia for Figma prototype testing, concept validation, and customer journey research, getting insights overnight instead of waiting for the next sprint. If your team wants to understand customers at the speed you ship products, try Strelia. Run your next study at strelia.io/lenny. That's S-T-R-E-L-L-A dot I-O slash Lenny.