Big Book of AI Metrics

**Peter Buchanan** (0:08)
Welcome to AI to ROI, The Big Story. I am Peter Buchanan with The New Plan. And with me as always is Ray Rike, the CEO of Benchmarkit. And Ray, today is a big day for us.

**Ray Rike** (0:21)
It really is, Peter. We've been building torches for quite a while. You've been prodding me to get this thing finished. And this thing is, we released the Big Book of AI Metrics today. And I want to talk about why we build it, who it's for, and how we think executives and leaders should use it.

**Peter Buchanan** (0:43)
Okay. So before we get in the book itself, let's frame why this moment matters and why the book is sort of needed. So the data related to AI implementations and ROI is not kind. You know, 27% of senior executives say AI's met their ROI expectations, that the average monthly AI token spend across enterprises has risen 13x since the beginning of last year, and most enterprises can't explain what they got for it. So it seems like enterprises and AI-native product companies need some help.

**Ray Rike** (1:25)
And you didn't even mention that famous now six-month-old MIT report that said 95% of AI projects are driving no return on.

**Peter Buchanan** (1:34)
I know. I had that in the report and I took it out because it's so last year, Ray.

**Ray Rike** (1:39)
I know. It is. Well, six months is definitely a lifetime ago. You know, it's funny there's this pattern that connects the successful AI initiatives and the failed ones. And it's almost always the same or at least grounded in a similar basis. And that is the companies that define their measurements of success, i.e. metrics, before they invest and deploy an AI initiative, typically will have the biggest win.
So a lot of companies say unfortunately, they're celebrating adoption and they're calling that success. So we wrote the book to close the gap between true success as measured on the income statement and balance sheet and AI adoption.

**Peter Buchanan** (2:28)
Right. That's the thesis. So let's get into it now, because for us, it's quite exciting. Yeah.

**Ray Rike** (2:33)
It is exciting, but I think we need to hear a word from our sponsor first.

**Peter Buchanan** (2:37)
Oh, that's right. We like sponsors. Sponsors are good.

**Ray Rike** (2:41)
Okay, Peter, let's dive into.

**Peter Buchanan** (2:44)
Ray, give listeners an overview. What exactly is the Big Book of AI Metrics?

**Ray Rike** (2:50)
Well, it is comprehensive. I think it's what, 180 pages and 81 different metrics. So it's comprehensive. It's very practitioner operator-focused reference guide.
It starts with an AI metrics and measurements framework for delivering successful AI use cases. Then we organize the metrics by use case type, by function, and even by stage of the AI lifecycle.

**Peter Buchanan** (3:22)
Right. So how's it different from what's out there? Because there are lots of frameworks out there, Gardner frameworks, McKinsey Playbooks. What do we do that's different?

**Ray Rike** (3:35)
Well, there's a lot of pontification that goes on social media sites like LinkedIn or even at conferences about here's how you should use AI. Here's a framework of how you should consider AI.
They tell you that measurements matter, but they don't go into any level of detail on what to measure, how to define it. How to ensure you have a baseline pre-AI versus the post-AI measurement, and even what a good result looks like. We built this Big Book of AI Metrics for operators and practitioners, not consultants and influencers.

**Peter Buchanan** (4:15)
Right. We've been publishing periodically the AI metric of the week in the AI to ROI newsletter. We've been doing it for months. Is this sort of those metrics compiled and organized or is it more?

**Ray Rike** (4:28)
Well, something I thought ever since I founded Benchmarkit was laser-like focus on one role. So a lot of the metrics that we kind of introduced in our newsletter, one was for us revenue leader, another one was for a product leader, a couple of them for CFOs.
But what I wanted to do is bring together one consolidated integrated approach and almost provide a systemic orientation of which metrics, for which function, for which stage of maturity in your AI initiative. So it's really an integrated systems thinking type book about metrics, starting with a really practical framework.

**Peter Buchanan** (5:19)
Right.
So you probably had a motivation for doing this because you spent your career as an operator. You've been in serious go-to-market roles. You've been a COO, you've been a CEO, you've had multiple exits, you've had benchmarked for the last six years, which does metrics and benchmarking based market research and builds measurement framework, helps B2B software companies build measurement framework. So what did you see happening in AI that looked familiar? And also, if you were an operator and this disappeared, the Big Book of Metrics appeared to you six years ago, ten years ago, and you're a CEO, how would you use this? So let's just go through a little bit of history, and then let's just say, how would this affect your life as a senior executive?