The Customer Service Revolution: Building Fin, with Eoghan McCabe & Fergal Reid of Intercom

**Erik Torenberg** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to share my conversation with Eoghan McCabe and Fergal Reid, CEO and Chief AI Officer at Intercom. Makers of FIN, the AI Customer Service Agent that's been a market leader since its launch some two and a half years ago. Regular listeners will know that Intercom has recently been a sponsor of the podcast. So it's worth noting that this episode was not part of that sponsorship deal. On the contrary, because I've been an Intercom customer for years at Weimark and also noticed that leading AI companies and past guests Anthropic, Gamma and Lovable all have testimonials on the FIN website, I wanted to understand what's really working and what remains a challenge for a company that's been among the most successful at creating practical business value with large language models. And I'm glad to say that this conversation really delivers. With a diverse customer base of more than 400,000 businesses, and Intercom's ability to measure successful resolution rate differences as small as a tenth of a percentage point, FIN is one of the most intensively tested large language model applications in the market today. And as you'll hear, Owen and Fergal are both remarkably candid, both about what they've learned and about what they still don't know. One perhaps surprising finding that stood out to me, especially considering how much the AI discourse tends to focus on new model releases and frontier capabilities, was Fergal's assessment that intelligence is no longer the limiting factor for customer service automation. On the contrary, he says that GPT-4 was already intelligent enough for the vast majority of customer service work, and that model improvements have only contributed a few of the more than 30 percentage point increase in resolution rate that the FIN team has delivered since launch. The vast majority of gains have actually come from better context engineering, which they have achieved through many rounds of careful optimization, of retrieval, re-ranking, prompting, and workflow design. Of course, we cover a lot more than that, including the fact that most customer service teams are currently underwater, which means that for now at least, FIN is allowing companies to support more customers and beginning to affect their hiring plans, but generally not yet leading to layoffs. How Intercom thinks about the importance of speed, and how they balance the desire to be first to market with the critical need to maintain customers' confidence. The culture of awareness, engagement, and constant experimentation that's allowed them to deliver that 1% improvement month after month for 30 months in a row. The intricate workflows that power FIN, and why Intercom is now training custom models for some tasks, including a custom re-ranker. How Intercom dogfoods FIN, and why their resolution rate, while above their customer average, is actually still quite a bit lower than top performers. Fergal's observation that no matter how sophisticated your offline evals, the messiness of real human interaction means there's no substitute for large scale A-B tests in production. How the 99 cents per resolution pricing model, which they pioneered, while initially unprofitable, has created strong alignment between Intercom and their customers and has become profitable thanks to improved success rates and lower inference costs. The 2x productivity goal that Intercom's CTO has set for their technology teams in light of AI coding assistance. And finally, how their vision is now expanding, from service agents to what Owen calls customer agents that can work across the entire customer life cycle, including sales and onboarding. Bottom line, if you're building AI products, you'll find in this conversation a bunch of valuable insights, from a team that has brought real rigor and sustained discipline to the challenge of making large language models work reliably for businesses and their customers at scale. This is Owen McCabe and Fergal Reid of Intercom.
Owen McCabe and Fergal Reid, CEO and Chief AI Officer at Intercom, makers of Fin. Welcome to The Cognitive Revolution.

**Fergal Reid** (3:52)
Thank you.

**Eoghan McCabe** (3:52)
Thank you.

**Erik Torenberg** (3:54)
So I'm excited for this conversation. My company, Waymark, has been a customer of Intercom for years. And so I've been following what you guys have been doing with AI with interest, both intellectual and applied over the last couple of years. And you've done some really interesting stuff and been, in some ways, really innovative leaders in the market. So excited to dig into all of that with you. I thought, first question, just because AI is moving so fast, and obviously you guys are running and sitting in the leadership position of an 1100-person organization that's distributed across all the time zones of the world, how are you going about keeping up with AI?