PP128: The Unseen Impacts of AI on Vulnerability Research

JJ (0:09)

Hey, everybody, we are at CyberSecCon. Welcome to Packet Protector, the podcast at the intersection of networking and security. I'm JJ, here with Drew Connery-Murray, and our guest at this particular moment is Mr. Trey Ford with Bug Crowd. Trey is going to be speaking here at the conference, but has thoughts and opinions on things, so we thought we'd drag him in here and ask a few questions about, actually, I'm going to read for everybody out there, I'm going to read the talk title because I think this is magnificent. Your session title is Before Midnight Tonight, Critical Thinking, Logical Fallacies, and the Urgency of Wisdom in the Age of AI, which I think we can all agree with.

The first thing I wanted to ask is, what are some of the fallacies that people are falling victim to? You've been around the industry for a long time, a lot of smart people.

Trey Ford (0:53)

Oh man, there's so much we can dig into. I think some of the biggest fallacies in play right now gets into first principle thinking. So, recency bias, sample bias, I think sample bias is probably one of the most terrifying. We're dealing with things that we grab ahold of. I don't know, it's like when you decide you're gonna buy a red Jeep, the only car you see on the road is a red Jeep. We're not asking first principle questions. So, we ask a question of an AI and any LLM, and whatever it gives us, great, we're gonna go to work on that. We don't stop and ask, what's missing in this picture? And so, I think a lot of folks are not pausing to reflect on where this come from, how was it trained, what should we be working on?

What's it missing? And so, I think it's important that we unpack that and challenge some of that, but we're making big corporate, very expensive decisions. And I don't think we're reflecting on the risk. And so...

JJ (1:42)

Now, do you mean this in the context of like the SOC and SOC analysts and the signals and the noise coming in, or do you mean this in terms of like the broader enterprise ecosystem, business decision making for all of the above?

Trey Ford (1:54)

Yes. I think I'll reject the charity of or. I think it's important to step back and say, whether it's the analysts, whether it's GRC, whether it's the folks that are harassing everyone through third-party risk management, if we're talking about the SOC, or we're talking about product security, security researchers, I think it's affecting absolutely everyone up to the C-suite as they're starting to augment their thinking, or I hate to call it outsourcing their thinking to these AI big brain systems.

JJ (2:18)

Where is the line between augmenting the thinking and outsourcing the thinking for you?

Trey Ford (2:23)

Intellectual honesty.

At what point are you maintaining authorship and defining what success looks like? What is the basis for a good foundation?

I think in cybersecurity, the analog would be the CISO is generally looking to drive decision making, and a lot of folks carried this as a load on their back. It was their problem. It was them against the world. The wise, mature CISO is partnering with the risk committee across the leadership of the organization, from sales to legal to marketing, marketing in a risk committee. They're bringing in all of the different executive leaders across the organization and pressure testing. Look, we have our risk register, here's our top loss scenarios. The things that we're all losing sleep over, is this rational? How does this impact the business? How do we prioritize this? And as a business, they make the decision together. When we hand that off to what we're going to call expert systems, LLMs, we've disconnected ourselves from owning that decision tree.

Drew (3:20)

So do you have examples of instances where you see people just entirely handing things over to LLMs as opposed to using it as an augmenter or a helper?

Trey Ford (3:28)

For sure. You'd start with the, okay, I'm starting my third-party risk program. Where should I start? And whatever comes out is where I go.

When do you talk to your security advisor, talk to your VAR, talk to your partners, talk to your peers?

We have an age-old joke where we talk about the beer ISAC. Trust between organizations is implicit, trust between humans is explicit. And so you can sit down and ask, how are things going? And you're going to share notes on, this is what hurts, this is what's happening, this is where we're getting killed. And when we start decomposing, where those incidents are coming from, where those disruptions or where the greatest exposures are coming from, life lived in other perspectives is what changes that. And so cognitive diversity, as opposed to single source decision sourcing and thinking outsourcing, there's a lifetime between those. It's a great starting place if you're going zero to one, but you've got to step back and really surround that with new perspectives.