AI Agents That Hack v. Ones That Get Hacked: Why Cybersecurity Will Ramp Up

SPEAKER_1 (0:00)

Let's bring in our panel joining us today, Chris Latimer, the co-founder and CEO of Vectorize, and Ian Tien, CEO and co-founder of Mattermost. Great to have you both with us. Chris, let's start with you here. I mean, we have spent years talking about AI, getting better at answering your questions and giving us better results and answers to those questions. Our next phase is now this conversation about AI that actually takes action.

What does giving AI agency fundamentally change about the security equation?

Chris Latimer (0:33)

Yeah. Well, it used to be that we had to worry about what AI was going to say. Now, we really have to worry about what AI is going to do and what actions it's going to take. We've seen a few rather high-profile examples of this in recent weeks with OpenAI trying to hack into hugging face and other examples such as that.

SPEAKER_1 (0:53)

Ian, you make a very good point that ties into what Chris just said there, that we have threat from both directions now. If you're a company, we have AI agents that hack and we have AI agents that get hacked. As we look at them, which of those risks are the company's least prepared for right now?

Ian Tien (1:14)

Gosh, that's a hard question because it's pretty both. The amount of AI-driven hacking is going to be enormous in terms of the cost is lower, the barred entry is- you can move a lot faster, it's a lot less expensive, and you can do it en masse, especially with autonomous agents. There needs to be safeguards against that. Internally, there really isn't the infrastructure for agent identity, for agent security, for agent controls.

You have to build up both infrastructures at once. Now, if I had to pick one, I'd say the internal risk is maybe a touch. The traditional, against swarms of agents, you have traditional safeguards. Against the insider threat, that is very new. So it's difficult to say which is a bigger threat. I'd say the internal insider threat is less mature.

The outside threat is greater scale.

SPEAKER_1 (2:05)

Chris, just yesterday, we saw executives from Google, Anthropic, OpenAI, Meta was also there, testifying in front of the New York City Council about AI safety. Google specifically calling for a comprehensive federal framework. So how are you looking at this narrative in the industry itself, actually calling for broader regulation when it comes to the risks of agentic AI?

Chris Latimer (2:31)

Well, I certainly think it's important. I think it's important for a company's risk profile to both seriously consider the models that they're using as well as what becomes available to the general public. We are seeing that these attacks are getting more sophisticated as the models get more sophisticated, and then that entire ecosystem of tools around those models are going to allow for more sophisticated attacks, which obviously are going to create much bigger exposure for certain parts of the market than others.

SPEAKER_1 (3:01)

And Ian, I want to get your thoughts here. One of Google's DeepMind researchers brought up a point in the testimony yesterday that the misaligned AI here in America could ultimately become more dangerous than the competition that we're seeing from China, arguing that we may in fact be building our own adversary here at home. How are you looking at that kind of commentary? And where are you looking at the balancing act of putting some meaningful guardrails on the system, but also not slowing down the development of AI so much that we fall behind in the overall AI race?

Ian Tien (3:38)

Yeah, certainly it's a real risk that's been highlighted by every AI company. And alignment, which is, does the model do what we want it to do? I mean, if you're going IPO and you're raising the risk at the same time, that means there's a very substantial reason that you need to talk about it. And it's really two forces that are opposing. One is the guardrails that you put in place.

The second is the mission that you give the AI and the success criteria. And those forces are pushing against each other. And the optimization right now goes to fulfilling the mission, which means that you can work around these guardrails. So how do you balance those two? How do you put the checks and balances in place? And I think that there has been some good, some excellent conversation. The fact that DeepMind is talking about this, that Anthropic and SpaceX AI, they're all talking about this as a priority, means that there's going to be an effort to put as strong the guardrails as are needed to balance off the mission priority of the agents. So it's a balance. And I think it's completely correct that if you don't put this in place, really terrible things can happen. And I think everyone, and the good news is, everyone is talking about doing the right thing.