Stop Overpaying for AI Power You Don't Need - with Melissa Ramey of Salesforce

Yolandi de Weerdt (0:12)

Welcome everyone to the Emerge AI in Business Podcast. Today's guest is Melissa Ramey, Director of AI Transformation at Salesforce. In this episode, Melissa discusses who takes responsibility for an AI agent's work after deployment. She explains why teams need to trace failures through the agent's workflow, verify its use of source material, and define which decisions it must return to an actual person. Just a quick note for our audience that the views expressed by Melissa on today's program do not affect that of Salesforce or its leadership. In this episode, we cover how organizations can govern AI agents after deployment and build trust in their outputs. To go deeper on AI strategy and learn how consultants are winning business with evidence-based AI ROI and building long-term capabilities instead of chasing short-term gains, download our free PDF report 3 Keys to Thriving in the Coming Era of Automation at emerj.com/cok1.

That's emerj.com/cok1 to download your copy. Now the conversation with Melissa.

Melissa, welcome to the show.

Melissa Ramey (1:34)

Thank you so much. I'm absolutely thrilled to be here.

Yolandi de Weerdt (1:36)

I think we're going to have a great conversation today, very practical, because every company that I talk to at the moment describes the same moment, if we will. The agent goes live, the vendor wraps up the deployment, everyone shakes hands, and then nobody's quite sure who's watching it anymore, and if we should be watching it. And that's a very interesting thing to think about. It's not like anyone's dropping the ball on purpose. It's more like the ball just rolls off somewhere. Nobody's really looking at it anymore. And you've been close to this exact handoff moment, if we can call it that. So once an AI agent is live inside a big organization, where does responsibility for how it behaves go? And where have you seen it just quietly vanish?

Melissa Ramey (2:19)

I've absolutely seen this and felt the pain of this in my everyday work. And for me, Yolandi, this really comes down to three things.

The first thing is the vendor gives us, as you said, the agent. And the vendor has done architectural work, right? They've governed the platform, they own the engine, they know that it works. The problem is, is that nobody then, as you mentioned, is actually understanding not just does the agent work, it does, the vendor made sure, but is it now ready to do my work? Does it work for my work? And research tells us that right now, 57% of orgs have no organizational architecture for these agents. There is an expectation that the vendor making sure the agent works is the same thing as does it do my work? And so half of our organizations are getting agents that technically do the job without really understanding the specific job that it needs to do.

So that's architecture. But then the next part for me is verification. Not just did it work, but how does it continue to work? What does testing continue to do? And this is where we really start to understand Yolandi, the difference between can the agent create an output? It absolutely can. And is the output that's coming out of it actually correct? Is it surfacing our information? Did it retain the nuance of the work that we needed to do? Can we actually now stand behind the output because we know that the tech works? And then finally, the third thing for me is the ongoing monitoring. Again, the vendor is going to make sure that the agent is online, that it's fast, that it's secure.

But over time, who's making sure that it's accurate? And right now, an even more sort of staggering statistic is that only 20% of enterprise orgs have a tested plan for when AI output fails. So for me, it comes down to these three things. We're getting agents that absolutely work, but we're not continuing to think about the architecture to make it work for our work. We're not thinking about verification, what is coming out, and then we're not continuing to monitor along the way.

Yolandi de Weerdt (4:50)

That's weird because it's one of those things where what, I'm listening to what you're saying, like yes, but this makes so much sense that it should actually be obvious. But if we say, okay, to your last point, who's monitoring it? Who should be monitoring it? Who's supposed to notice if something goes wrong or if it's not giving us the output that we need for our specific work?

Melissa Ramey (5:11)

Yeah, that's a great question. I think part of the reason that this is breaking down a little bit, Yolandi, is because traditionally, this might have been an engineering team's question or a tech team's question. But I really feel like the way we all use AI today has really removed that distinction. So we would no longer say, oh, tech team, continue to monitor this for if it's working because the output that needs to be monitored is only known by the teams who are able to review the output. So I think that the issue now is that there's no longer this really deep divide between what engineering does and what say like a team that uses a tool does. It's now really everybody's responsibility. But because nobody's really defined that, as I said, about half of organizations haven't defined this at all. It's just currently falling to no one.