Why AI Agents Break the GenAI Security Model with Devvret Rishi

**Sam Charrington** (0:00)
I recently attended a major enterprise tech conference, speaking on a couple of panels about scaling AI agents. Naturally, the topic of risk came up a lot, and the default answer was usually some combination of static guardrails and human approval. In theory, this sounds simple enough. Block the dangerous stuff, and when something looks risky, put a human in the loop. But agents put pressure on both sides of this model. Static rules are hard because agents are creative. They don't just follow a fixed path through software, they plan, improvise, call tools, and find workarounds.
And human approval is hard because agents can operate much faster than we can. So the question isn't whether we need guardrails and oversight. Of course we do. The question is what that should look like when agents are operating at scale across high stakes tools, databases, and workflows.
I spoke about this with Dev Rishi, GM of AI at Rubrik, where he and his team are building infrastructure to secure and govern agents in enterprise environments. He shared an example from his personal experience that gets at why this is such a tricky problem.

**Dev Rishi** (1:05)
But then we saw some more sophisticated types of things that were going wrong. One thing we noticed was that Cloud Code was really trying to post internal source code to a public repo rather than private. And so we saw this kind of relatively frequently, and there was even instances where like, if we looked back through the audit logs, we saw like the check, check, check. And so technically it felt that it had gotten the human in the loop approval for it. And we even saw one crazy instance where Cloud Code tried to get around this blocking we were doing of like GitHub public gist.
And rather than like doing this as a text in text out system where it was like, all right, I'm posting to this URL, it spun up a browser window and we just started to see mouse clicks on certain coordinates. And we noticed that one of the coordinates actually was for a public gist.

**Sam Charrington** (1:50)
I'm Sam Charrington and this is The TWIML AI Podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next and what matters. Let's jump in.
Is it a learning challenge fundamentally, or is it an expertise challenge? Yeah, there's lots of ways to think of the challenges, but it does seem to be, to a large degree, kind of mindset and a mindset shift that's required.

**Dev Rishi** (2:30)
You know, Sam, I do think like a lot of other tech changes, there is like, there's a cultural component, there's a learning component, but I actually don't think that's the biggest piece.
I actually think the biggest piece, after speaking with a lot of the organizations now, is the approach and how to manage risk. Like, if I thought about what's different from a fast moving AI native organization and what's different from, let's say, a top 10 global bank or a top 20 health care company, the way that the health care company and the bank have really been brought up is that they have to make sure that the ways and the systems that they have are deterministic, that they have guardrails, that there's real clear downside protection.
If you've used an agent like Clawdcode or Codex before, doesn't exactly feel like there's a ton of downside protection as you're using it all the time, right? It's incredible. Like I had one global CIO describe it as like, it feels like a fast car with no breaks. Like I'm moving really, really quickly, and you know, who knows what exactly is happening on the background. And I sympathize with that. I think the biggest thing is that these tools have come out with an incredible capacity to be tools. But they didn't really come out with like a perfect way to secure and govern them. And they're really operating on legacy IT infrastructure.
Honestly, the biggest difference I see between the startups and these like global 2000 enterprises, the global 2000 enterprise has maybe a bit more of a legacy mind shift, but really more so just more to lose. Like as they take kind of take this approach. And that has been like the number one thing that's hindered the AI adoption. So if you're an AI startup, you can use one of these harnesses out of the shelf and start, yeah, going on it. If you're an enterprise, you're going to probably bog it down with some AI governance committee meetings. You know, essentially, let's meet this week to decide, define a framework three months later to come up with a V2 of the framework and so forth. And that's what ends up, I think, delaying the site.