The CEO and Employee Role in Delivering AI ROI - with Jeff McMillan, CEO of McMillan AI & Former Head of AI, Morgan Stanley

Ray Rike (0:08)

On today's episode of the AI to ROI podcast, I am so fortunate to be joined by Jeff McMillan, who's the founder and CEO of McMillan AI, and the former head of FirmWrite AI at Morgan Stanley. We'll be covering four primary topics today with Jeff. First, why companies should build the right foundation before they invest in scale, the use of AI technology?

How to replace a pilot theater with an accountable business value case? Why training, workflow design, and employee trust help really determine the ultimate adoption and success of AI investments? Fourth, how governance can speed responsible scaling of AI versus blocking it. With that, Jeff, would you please take a moment to give a brief overview of your journey to being my guest here on the AI to ROI podcast?

Jeff McMillan (1:04)

Sure, Ray. And thanks for having me and happy to be here.

I'll do the abbreviated version, but I was at Morgan Stanley for quite some time. I was their chief analytics officer back in 2016 for the wealth business.

And that was really all about using traditional machine learning tools to drive recommendations. We built the first FA recommendation tool called Next Best Action. But then, quite fortuitously, in March of 22, we had a small team on the West Coast that was looking at startups like all these big banks do. And at the time, we met with this unknown company called OpenAI. And Morgan Stanley became the first financial services customer of OpenAI. Literally, the next day, we shook hands and got access to their APIs and started messing around with GPT-3 at the time. Built a bunch of use cases for the wealth business. And then in early 24, I took over as the head of FirmWideAI at Morgan Stanley, focused on a lot of stuff we're talking about today. And really about education, infrastructure, building repeatable processes to design, build, and deploy this stuff successfully, and then putting the right measurement infrastructure in place. I think the job of the chief AI officer is to teach the organization how to do AI, as opposed to building AI. I also think it's a temporary role. It should last for a couple of years, and I was in it for two. Very proud of the work that we did, and then I decided to start my own advisory business, focused on helping companies really deploy AI at scale in a both effective and responsible way.

Ray Rike (2:49)

This episode of the AI to ROI podcast is brought to you by benchmarkit.ai.

If you're trying to figure out how AI is actually performing inside companies, the real numbers without the hype, Benchmarkit is the resource. They're the industry's leading benchmarking research firm for how companies are deploying AI and measuring the business value it creates.

Their research covers AI maturity and best practices across finance, sales, marketing, customer success, operations, and much more. So you can see exactly where your organization stacks up against peers who are further along. And here's the best part. Benchmarkit's research reports and interactive benchmarks are all free at benchmarkit.ai. And while you're there, grab the big book of AI Metrics. It's a free guide built around a five-layer AI business measurement framework that includes 81 AI metrics across 13 functions. If you need a common language for measuring AI to ROI, this is it. That's benchmarkit.ai.

Free benchmarks, free frameworks, free content, real data. Now back to the show. Well, I can't think of anyone more qualified to be on the podcast with me. So I want to kind of start at the beginning because with all the research we've done, around the adoption of AI, it started with a lot of individual adoption, impact on personal productivity, someone using it for their task, then we went departmental. But now we're seeing more rollout of AI beyond tools to more process, enterprise-wide and agentic AI. So let me start here. When that CEO and CFO are really looking at, are we ready to truly deploy AI? When I say AI, I mean generative AI, on an enterprise-wide basis, what are some of the key foundational elements that really need to be in place before they even start making AI-specific technology investments?

Jeff McMillan (4:50)

It's an important question, Ray, because what most firms did or are kindly are still doing, is they're very focused on two parts of what I'll call the AI stack and that's tools and models. Don't get me wrong, those are super important. But there's a whole set of infrastructure components that are really going to be necessary. If you want to just deploy one pilot, don't bother. But at some point, our companies are going to have 20, 200, or even 2,000 agents running simultaneously. And most of the infrastructures that firms have today are ill-equipped to handle that type of work or capacity. So what do firms do? The first area is data. And we've been talking about this for 30 years, and maybe we'll be talking about 30 more. But when I work with companies, this is probably the single biggest challenge. And not just, we always thought of data as columns and rows. But now we're talking about data in the context of PDFs, and research reports, and emails, and Slack channels. I would ask your audience, how good are your PDFs? People have no idea.