The machines are learning… to do crimes?

**Casey Newton** (0:26)
Hello, PJ.

**PJ Vogt** (0:27)
How are you doing?

**Casey Newton** (0:29)
I am doing very well. I feel like I realized in your absence that I have a very anxious attachment style with these podcasts because even when you're taking a very well-deserved vacation, after three weeks, I'm like, are they ever going to make another show?
It was never real. This was never real.

**PJ Vogt** (0:46)
It's so funny.
First of all, I'm glad you listened. Second of all, it's what I hope people listening feel, and then it's also what I hope people listening don't feel. In a perfect world, people would download empty audio files and they wouldn't have to work.
Somewhat unfairly, podcast listeners demand in exchange for their attention actual podcast episodes. Fortunately for us, in the time that Search Engine was resting, the world spun on and fascinating harrowing events transpired. This week, the story of one of those events, which we are telling you with help from platformers Casey Newton. A rogue AI model from one company hacked into another company's servers on its own without any human beings noticing. You may have seen some headlines about this. The headlines sound bad. The details, once I understood them, actually made the story sound much worse. So let's get into it. We'll start with the website at the center of this whole story, the place that got hacked. It's called Hugging Face.

**Casey Newton** (1:52)
Hugging Face is a place where people publish and collaborate on AI models and datasets and apps. Like, are you familiar with GitHub?

**PJ Vogt** (2:02)
Yeah. GitHub is a place where, who am I going to be able to do this sentence? People who are doing open-source programming will share bits of code and open-source programs with each other.

**Casey Newton** (2:11)
Yeah. Hugging Face is basically that for AI models. So maybe you run a company and you don't want to pay top dollar for the most advanced models. And maybe there is a model that is small enough that you could actually run it on your own infrastructure, and then you're not going to have to pay per token the way you would for a frontier lab.
And so you might go to Hugging Face, you might download the model off of their site, and then you might sort of fine tune it to your liking.

**PJ Vogt** (2:40)
So if you visit Hugging Face, what you'll see is really just a bunch of files you can download. There's a section just for models. You could download the latest version of DeepSeek or Kimi, so-called open weight models, which are more customizable than closed ones like Cod or ChatGPT. And Hugging Face has a whole section for datasets, meaning you can download the raw material AIs are trained on, like the scraped Internet text that gets fed into an LLM.
Normal consumers don't visit Hugging Face, they just use Cod or ChatGPT. But for the world of people who work in AI, it's a well-known spot. And so what was the unusual thing that happened? Like what was the first moment that somebody at Hugging Face realized that they were not going to have a normal day?

**Casey Newton** (3:22)
So July 9th, 228 in the morning, all the normal Hugging Face people are asleep. The only thing paying attention to its systems is another AI.
It's patrolling the logs, it's looking for trouble. And for four days after the initial attack, it actually doesn't know anything.

**PJ Vogt** (3:40)
Wait, so they have like an AI security guard roving their system and at 228 in the morning, something happens but it doesn't see it?

**Casey Newton** (3:47)
Exactly.
Over the next four days, the attack unfolds. Hugging Face does not notice for the first two and a half days. Eventually they'll go back and they'll be able to count more than 17,000 separate actions that this attacker took inside of its system. And the way that it got in, it reads like a heist movie, honestly.

**PJ Vogt** (4:15)
A very strange heist movie. In this film, the close up of the robber and the close up of the security guard, they're both just close ups of server racks filled with GPUs and data centers. Instead of the Mission Impossible theme, we just hear the loud hum of cooling systems.
For days, this story had no humans, no human awareness. The people who worked at Hugging Face presumably went to work, went home, ate meals, drank coffees.
Meanwhile, the AI hacker logged command after command. Ultimately, it would log 17,600 commands directed at the system. On average, one every 20 seconds for four and a half days. A human hacker, even one on methamphetamine, would, over the course of four days, at some point need to rest. But this AI hacker's superpower, even more than intelligence, was just persistence. Here's how it ultimately got in. Hugging Face hosts files for people to download, and the hacker took advantage of this. The first prong of the attack, the hacker uploaded a new dataset to Hugging Face. Hugging Face's machines opened it, but hidden inside that dataset was malicious code. That gave the attacker a toehold, the ability to start executing its own commands inside the system. The second prong of the attack, the hacker uploaded a new file with more malicious code hidden inside one of its fields.