Why Most Enterprise AI Pilots Fail: Lessons on Trust and Deployment

**Shub Agarwal** (0:00)
I think the fundamental problem the organizations are facing is they don't know how to take those demo to production. And that is where I saw the gap.

**Cindi Howson** (0:21)
Hi, Data and AI friends. We are at a generational shift. Some might even say a once-in-a-lifetime opportunity. To capitalize on this opportunity, you need more than just great technology. You need a mindset shift. That's why I'm proud to say that ThoughtSpot sponsors this podcast. Go from raw data to trusted insights and action with ThoughtSpot Agentic Analytics, a platform powered by a suite of specialized AI agents. It's why companies like Cisco, Lyft, T-Mobile, Sephora and Schneider Electric count on ThoughtSpot. See what the future of analytics looks like at thoughtspot.com.
Welcome to The Data & AI Chief. I'm your host, Cindi Howson. What if the biggest barrier to AI in your organization isn't the technology, but rather the organization itself? Too many companies are sitting on a graveyard of AI pilots and promising demos that never made it to production. Here to help us figure out why this is happening and what to do about it is Shub Agarwal, founder of the AI Trust Lab at USC and author of Successful AI Product Creation and Nine-Step Framework.
Shub has spent two decades in product management and AI leadership at companies such as Amazon, Silicon Valley startups and Fortune 50 companies. He now bridges that experience with his research and teaching at USC.
In this episode, you'll learn what's actually preventing AI from scaling inside most organizations, and how to think like an AI organizational leader rather than just a builder, and what it really takes to make enterprise AI trustworthy. Shub, welcome to The Data & AI Chief.

**Shub Agarwal** (2:34)
Thank you, Cindy. Thank you for inviting. I'm super excited to be here.

**Cindi Howson** (2:38)
Yes, I am too. We had the pleasure of meeting in person coming up on a year ago in Santa Clara, at Gen AI Week. Where in the world are you joining us from today?

**Shub Agarwal** (2:52)
I am in Los Angeles, where I live.
I teach at USC, so I'm very close to that. This area is called Rancho Palo Verde.

**Cindi Howson** (3:01)
Oh, very nice. So let's start out by talking a little bit about your book, Successful AI Product Creation.
What problem were you trying to solve in writing this book now?

**Shub Agarwal** (3:15)
So I came in in this industry in the AI field from a long time ago, and what I have seen is the teams that are struggling with building products in a systematic way, AI products in a systematic way, because AI changed the definition of working, and so teams didn't had a mental model and a framework to build AI products in a systematic way that are successful. And that is why what you see today, the 80 percent demo and 20 percent production deployments. That's what the study is. Ninety percent of the AI projects are not showing ROI.
So, I wrote this book to bring some system and rigor to building AI products. And my hope is we can flip that 80 percent demo, 20 percent production to 20 percent demo, 80 percent production.

**Cindi Howson** (4:12)
Yeah. And do you think maybe that this is just a learning curve that organizations have to go through because there's so much new. So, maybe they have to experiment first before they can step back and say, aha, now I see how we can use this.

**Shub Agarwal** (4:35)
I think experimentation is very, very critical and very, very important. We have to experiment. We have to try. We have to learn.
That is there and that will always remain. I think the fundamental problem the organizations are facing today and the teams are facing today, is not that they have lack of experimentation in the demo aspect. The challenge is they don't know how to take those demo to production.
And that is where I saw the gap. And that is what we want to fill the gap with, providing a system and a method that team can apply on a rigorous basis, on a regular basis.

**Cindi Howson** (5:18)
Yeah, great. Now, you offer a definition in your book that I really liked. The product manager's job is to clearly define the business problem and assess whether AI provides a unique advantage over existing solutions, whether human or rule-based AI systems, while delivering tangible business value.
How well do you think that most product managers are able to focus on that business value rather than this is a fascinating new technology?

**Shub Agarwal** (6:01)
I think thinking of for the product managers, if they have to think of the business value and the outcome, I think it is very critical for them to understand what the end goals are and how do they attach those end goals and then think and work backwards to build a solution from it. Lot of product managers today, what they do is they think of AI as a tool and then try to solve and implement that tool. So they build AI for the sake of AI instead of trying to solve a real business problem.