Aaron Levie, Steven Sinofsky & Martin Casado: How Do You Secure a World of AI Agents?

Steven Sinofsky (0:00)

If you regulate AI too early, you actually don't solve anything. You still just kind of had the same risk ultimately.

Martin Casado (0:06)

You willed the thing into being, but you haven't figured out how to control it.

Steven Sinofsky (0:09)

The problem we have now is this rift between the labs and the security community that keeps coming to two conclusions. Sloppy, and you're not complete in what you're telling us happened.

Martin Casado (0:20)

An employee is like 10 percent chance of species extinction. The post is very reasonable, but the atmospherics are not.

Steven Sinofsky (0:28)

Agent swarms completely flip that. These are just roaming drones, but like time 10,000 and they will easily mistake a good task for a bad one. So now we need a whole layer internally that just is tracking way more about what authentications are being done, what APIs are being done. The US about 15 years ago stopped leading in tech antitrust. The problem is that Europe is going to lead with that because they have nothing to lose.

Martin Casado (0:51)

This could change the nature of software fundamentally.

Steven Sinofsky (0:54)

The center of innovation has just moved.

Erik Torenberg (0:56)

This is the signal that the early Internet was riddled with viruses, worms and security failures. We didn't stop building it. We learned how to make it safer. What should I take from that history?

In this episode, I sit down with Aaron Levie, Steven Sinofsky and Martin Casado to debate AI safety regulation and what changes when agents start operating across the software we use every day.

We get into why agents could force a rethink of permissions and cybersecurity. What decades of software security can teach today's AI labs, and why regulating a technology before we understand how it actually fails, can create problems of its own. And we look at a broader shift already underway. As models mature, some of the most important AI innovation may increasingly happen outside the frontier labs, in the systems and software built around them.

Guys, welcome back to the podcast.

Martin Casado (1:51)

Thank you.

Steven Sinofsky (1:53)

Didn't think we'd ever do this again.

Martin Casado (1:55)

I can't believe this is great.

Steven Sinofsky (1:57)

I mean, Martin's just building these $100 billion companies. Too busy for this podcast.

Martin Casado (2:01)

Or at least taking credit for it, as VCs do.

Erik Torenberg (2:03)

Exactly. We've a lot to discuss today, but Aaron, we'll only start with you.

Pacing the frontier, how have you reacted and reflected on what's happened there and just the discourse that's followed?

Martin Casado (2:13)

Oh, boy.

Steven Sinofsky (2:13)

I think we should start with Martin on this one. You were fighting lots of good ground wars. Maybe I'll say one thing that we probably all agree with, and then we can figure out where we maybe kind of fracture off.

I think we would agree that any AI lab right now at the frontier should be building in the safest way possible with the highest degree of governance and security and whatever your definition of alignment is. This is an incredibly important area of research. It's an incredibly important area for the diffusion of AI. You're not going to have AI diffusion without extremely high quality products that can be trusted by enterprises and that aren't constantly hacking systems.

When at least I read the Dario post, I actually didn't disagree with almost anything because it was all about how do you have better security of these systems, sandboxing, better testing. There's going to be some debates around the embedded nature of the testers. And do you agree with who those are and does the industry all align on that? But I think actually all of the major points were probably salient and appropriate. Then the only question is, does this get sort of user leveraged to do things that maybe we don't agree with? Which would be like a slowdown of AI dramatically because of regulatory controls that would sort of not make it easy to compete with the frontier labs? Or do politicians kind of end up sort of taking the message and run with it, and maybe even worse outcomes happen? It's used to ban data centers far faster and whatnot. And so I think the actual substance of the topic is actually incredibly important, and I think very important for AI advancement in general. And then the question is, what do you do about it? Especially what do you do about it from a regulatory standpoint? And that's probably where the industry is going to land on very different points in the continuum. And I think that Martin was putting up a good fight on, let's make sure that we don't use this for regulatory kind of capture, which I also agree with. But I think the ideas in the pacing conversation are important. Again, it's a little bit of a funny concept, because maybe it's not even pacing as much as just good hygiene. And engineering, and so with good engineering, obviously there is a slight slowdown, but it's a slowdown that obviously allows acceleration of your diffusion, because you wouldn't be able to have any of the AI be diffused if nobody would trust using it.