Salesforce’s Rohan Kumar on the Unsolved Problem Holding AI Agents Back

SPEAKER_1 (0:00)

Since we started our series, The Org Age of AI, Wills Kang and I have been exploring a problem that has always been there, but no one really needed to solve it. Businesses have endless data, enormous amount of it, but they've never recorded the context that makes AI truly useful. The human thinking AI needs to learn from is still largely in humans' heads. In one of the articles, we put it this way, you know the knowledge exists, but it's stored in people and not in the systems. Think about how decisions are made. You check with your boss, something needs to be done, an exception, and she says yes to this exception, and then you and your team moves on. You might record the outcome into the database, but the reasoning behind this decision often stays with you and your boss in that situation. Now, an agent is supposed to understand how this whole process, how this whole mess works across thousands of people and years of accumulated non-breeding decisions. We've called the work of making that knowledge usable organizational translation, and I wanted to pick Rohan Kumar's brain about this. He spent more than two decades at Microsoft, including leading Azure Data, and now he is Salesforce president and chief platform and engineering officer. He needs to actually solve all these problems, and he's worked on the systems company used to store and manage their data. Now he is building infrastructure for agents that are supposed to act on it. His comparison between coding and business decisions gets to the heart of the problem. What do people leave behind that a model can learn from? And what does capturing that knowledge require of the engineers building the systems? We begin with the infrastructure around the agent, often called the harness. How you test it, decide what it can access, and give it the context to do its job. It's a short but very insightful conversation. From the engineering perspective, can you discuss, explore the difference between building for engineers and then working more with agents?

Rohan Kumar (2:08)

If you basically look at a typical agent life cycle, just like anything else, you write an agent and there are certain sort of evals that you basically have, like typically they're called the golden set, but you know what the answers should be for certain kinds of questions that get asked. And then you measure the efficacy of the agent based on those answers. But people define the harness, they typically define it that what's the runtime where the agent runs and what are the APIs that they call and then the responses that they get and how do you measure the, whether the responses are correct or not. The approach we've taken at Salesforce, when we talk about the enterprise AI harness, it's much broader than that. So there is the capabilities for running the agent, authoring the agent, testing it against evals, but then how do you manage those agents?

What I mean by that is, there has to be a central place in the enterprise, where you should be able to scan as new agents come out, and existing agents get deactivated. There has to be a continuous scanning that happens because you need an enterprise-wide registry. So that's a capability. Then you essentially think about, how do you secure these agents? Today, a lot of these agents that are getting authored by engineers, they are running on behalf of somebody, right? So the identity of the agent essentially takes on the identity of the user using the agent. But that's a problem, because now you've essentially exposed the agent to the same privileges that the user's identity has. So there is this whole focus on creating an identity that's very specific for the agent and limiting the privileges of that identity. So they cannot do, even if they go rogue, they shouldn't be able to take your confidential data out of the enterprise and send it somewhere else, right? Then there is, if you take a look at data, there is a focus around what I call data security. The most important asset in any enterprise is their data. From there, the knowledge is getting created. But more, if you look at it, the challenge is, if you don't understand, if your data is highly confidential, and if it's highly confidential, what can the agents do with it? All that has to be very governed AI policies, right? Which is again, outside of just the building of the agent. And the final thing I feel essentially is, ultimately, if you look at how efficient the agents are. So today, one of the biggest challenges, have you heard, you know, you've heard of token maxing, right? Like, you spend all this money, you can, the brute force effort that sometimes we do in engineering is, okay, all this is my data corpus. I'll build connectors to all of them, whether it is SaaS applications, data warehouses, data lakes, and then I'll give a prompt to the agent and other model and they go figure it out.