An Argument Against AI Doom

SPEAKER_1 (0:00)

Tired of rage scrolling through AI Slop and Clickbait just to get a handle on today's news? Meet InPress, the algorithm-free news app that makes it fun and rewarding to stay informed. Track your media literacy and rediscover reality. Download today at InPress.app. That's I, N as in news nerd, press.

SPEAKER_2 (0:21)

This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required compatibility and availability varies 18 plus.

Zack Korman (0:46)

I think that realistically what will happen is models will get better, and if the labs do not up their cybersecurity game, and if the company's running AI agents do not up their cybersecurity game, they will cause real harm. And then we will like literally get them in trouble. Like we will put someone in prison because you killed people. You're not allowed to kill people. And the do-mers would say no, but then it's too late because they're afraid that the people we killed would be like 8 billion.

Whereas I'm not.

I'm like no, because that's not the real world doesn't work like this.

Charlie Warzel (1:23)

I'm Charlie Warzel, and this is Galaxy Brain, a show where today we're going to calibrate our anxiety about the AI hacking epidemic. Specifically, we're going to answer the question, is the rogue AI agent problem really just a normal cybersecurity issue, or is it something else altogether? It's a scary moment. As we're recording this on September 29th, the news is moving quite fast. But allow me to summarize some of the broader points. Since August, there's been a pretty steady drumbeat of revelations starting with OpenAI's agents hacking hugging face during a training exercise. A few more followed. Agents hacked into a German website to use it as a rudimentary message board. An agent for OpenAI gained unauthorized access to a data portal that was run by Australia's Universal Healthcare Insurance Provider.

And last week, OpenAI revealed that its models attempted to hack or otherwise access numerous US government websites.

That same day, OpenAI announced that 53 incidences happened where photos were uploaded by users to the company via its products, which were then uploaded without permission onto other platforms. Last weekend, Axios reported that OpenAI and Anthropic were looking at tens of thousands of potential incidences where agents were behaving in unexpected ways, either coordinating or trying to get around or succeeding in getting around company guardrails and causing problems on the Open Internet. Now, there are some positive signs here. This week, OpenAI announced it was pausing training of its most capable models and delaying the release of its newest model, citing security concerns. But let's be clear, these are very real examples of technologists losing control or being unable to monitor their tools. And there's lots of talk about what kind of liability these companies ought to face. Whether these types of hacks are subject to penalties under the Computer Fraud and Abuse Act. Online, there's been a constant debate over what is going on, why nobody has been held accountable, what accountability means, and what needs to happen next.

But all that chatter and all this news has merged with a different narrative online, the one that is best represented by the AI safety and Doomer camp and the fears that an out of control, self-improving AI system could one day, quote, kill us all. But these are in a way two different stories. One is about a hypothetical improved set of models. It's a frightening scenario, but it is speculative. The second story, the one of the AI agent hacks is not speculative. It is something that is happening right now, and it is both urgent and it is tangible. And my guest today, Zack Korman, argues that this is a problem that is not sci-fi in nature. That we are not dealing with machine gods, with agency, but with a sprawling cybersecurity problem. And the companies are acting negligently.

Korman runs Embroidery, an AI agent monitoring and detection platform. And so I wanted to have him on to talk about what's really going on here with these high-profile hacks. Can we constrain these agents? If so, how? What are the real risks involved with agentic AI? And how do we keep them from coming to pass? Korman joins me now to talk about it all.

All right, Zack, welcome to Galaxy Brain.