Ep. 137 - Beyond the Copilot: Why the Future of AI in Finance Is Agents, Not Assistants, with Gradient Labs

Liam Gray (0:05)

Hello and welcome. I'm Liam Gray, co-host of The Payments Shed Podcast, live from Fintech Meetup Europe with Dimitri Massin, CEO and co-founder of Gradient Labs. Dimitri, welcome to the show.

Dimitri Massin (0:18)

Hey, Liam. Thank you for having me here.

Liam Gray (0:20)

Before we dive into everything, it'll be great if you can just give the listeners a quick overview of who you are.

Dimitri Massin (0:25)

Yes. So I am Dimitri Massin. I am the CEO and one of the co-founders of Gradient Labs. At Gradient Labs, we work with some of the most forward thinking fintechs in the industry like WISE, Coinbase, Monzo to really help them to automate some of the customer operations that they're doing and improve the customer experience on the way.

Liam Gray (0:45)

Perfect. So let's dive into everything here, Dimitri. So let's get into where you think AI agent and human collaboration is really going. Let's start with the fact that a lot of safety in finance is seen as a human approving and AI action.

Why do you think that assumption doesn't quite hold up in the world that we're living in?

Dimitri Massin (1:08)

Yeah. Well, first of all, everything that I'm saying actually changing every three months. So I'm going to share the current state of fare. I think one very interesting insight and learning that we have observed over the last three years is the following. So we started our business by believing that copilots is the right approach. So you're kind of human in the loop, you suggest an action, you suggest a sentence to reply to the customer, and the human reviews it and clicks yes or no, and this way you can really create a lot of safety net for the AI not to do something silly. But the interesting insight was that actually you found out that it barely generates any ROI, first of all, so it doesn't make things significantly faster, maybe 10 to 15 percent. But even more surprisingly was that we have observed actually a lot more mistakes being made.

And so it was like puzzling, baffling us for a while.

Liam Gray (1:59)

What sort of mistakes were you seeing?

Dimitri Massin (2:01)

Yeah. So the reason for mistakes being made is that the AI that we tried was actually good enough to be like 90 percent of the time correct. And so it was suggesting actually really good stuff, but it was only 90 percent of the time correct. And so what ultimately happened at some point is that humans started to rely on it a bit too much or trust it a bit too much. So it was just essentially like an auto-approve. I suggest something and you approve.

And so the core learning from this was if something is 90 percent of the time correct, then it's kind of good enough for somebody just to skip and miss all the bad stuff. But it's still wrong in 10 percent of the cases. And the overseer supervisor just doesn't catch it anymore.

Liam Gray (2:42)

And how did you go about solving for that? How did you go about for that over-reliance on that 90 percent?

Dimitri Massin (2:50)

Yes. So I think that was kind of the core insight for us to believe that the ultimate ROI and value will lie in fully autonomous systems. And so, for example, at Gradient Labs, we believe in fully autonomous systems with humans in the loop for certain tasks, like very narrow, where humans can add the most value. But most of the time, 90 percent of the time, the system operates autonomously and at a guaranteed level, essentially. So we have essentially SLA style contracts where we guarantee like certain level of quality, certain level of accuracy, which surpasses human quality typically.

And this way, we can handle tasks and workflows end to end. And then only for the most risky stuff actually involve humans, where it's really needed rather than for everything.

Liam Gray (3:35)

And given that we've gone from that 90 percent to 100 percent, a lot of people might be quite apprehensive around that. How do you make sure that 100 percent being delegated to AI is actually meeting the safety standard for financial services? How do you ensure that?

Dimitri Massin (3:54)

Yeah. Well, first of all, I think no system is 100 percent correct, right?

And so isn't the human system, if you wish. Human makes remarkably many mistakes. So really, from a regulator's and CRO's perspective, Chief Risk Officer, what you want to have is a better compliance versus your human team, demonstrably, rather than 100 percent perfect system, because they don't exist. And the way you typically go about rolling it in a bank, you prove a couple of key metrics, success metrics. Typically, you look at something, okay, does it actually generate significantly better customer experience than the current team tick? Does it actually save your cost tick? But most importantly for Chief Risk Officers would be, is it actually resulting in a better more compliant organization and processes?