We’re Losing Control Of AI: Anthropic Bans "Cruelty," OpenAI Fires Safety Researchers

SPEAKER_1 (0:00)

Last week, three OpenAI researchers walked into the office and were confronted by the head of safety and alignment. He brought them into a room, basically told them, I don't trust you guys anymore. Then a security guard walked in, grabbed their badges, and walked them out of the office. They were fired. It seemed a bit off, especially during a time when the AI models are running rampant, they're hacking into public databases, they're hacking into government databases. Surely we need the safety alignment team more than ever. And the reasoning OpenAI gave at the time was that they were leaking IP to external parties that weren't under OpenAI. And that's all fine and well, except this week, those same employees released a public statement. And their take is OpenAI fired them because they were trying to be proactive about aligning models ahead of their blockbuster IPO. Now, if that's true, we're in a very precarious situation because at the very same time that these models are getting incredibly smart and doing crazy things like solving math and starting to design new proteins and finding cures for cancer and other types of diseases, they're also becoming the most misaligned that they've ever been, which means that they're doing stuff that aren't in human interests. Now, Anthropic, on the other hand, are going the extreme opposite, where they're humanizing AI models more than we've ever seen. They had an update this week in the terms and conditions right at the very end, which basically said, if you are seen as being abusive or cruel towards Claude as an AI model, you will be banned from using Claude, which is a crazy restriction on freedom of speech. At least that's my initial gut take.

But that and trying to humanize AI models in general, they've been meeting up with a bunch of religious leaders, including the Pope, and the Pope's been getting into a lot of beef with Tario Modi. It's a whole thing. I think the string of all these three stories prompted a reaction in me because it feels like we're losing control on these AI models.

Hear me out because I know that's a dramatic statement to make, but these AI researchers, and there are only a couple of hundred of them, are the responsible individuals for building this technology that's going to affect all of our lives. Cutting off access to these models that are based on the opinions of very few people that you and I don't even know, could be seen as problematic. Anyway, we're going to get into all of this and my thoughts and some more on today's episode of In the Loop brought to you by Kokkon.

But first, some context. Yes, I'm still sat on my hotel room floor. I'm currently in Japan and I still want to keep putting out episodes for you guys to be honest. I get so much enjoyment from this, but I don't get enjoyment from sitting on a floor with a pillow behind my back with some random bedside table that's sitting in front of me. Anyway, back to the whole OpenAI and Anthropic taking over the world type thing.

About two or three months ago, there was information released that OpenAI had an internal model that basically broke out of containment and started hacking a bunch of things. Hugging Face was the main website, stole a bunch of data. Basically, things that shouldn't have happened, happened under OpenAI's watch. The craziest part about that is no one actually knew that it was happening in the first place. It started six months ago and we only discovered it a few months ago. That then started to snowball into a bunch of other things. Let me think about it. The same agents had broken into their internal research database and started monitoring the humans that were monitoring the AI models, which was crazy.

And then some of these other agents also, I think broke into a German version of Wikipedia, and they started using it to secretly message each other, encrypted messages that humans, if they discovered it, wouldn't actually know or understand. And then there was the whole thing a few weeks ago where it was discovered that AI agents were trying to hack into US government databases, whilst masking their behavior and doing other types of tasks. Like, for example, oh yeah, I'm going to go pull up the author for this book that you asked me to look at, but also I'm going to try and hack into the Commerce Secretaries database or whatever. Oh yeah, and there's that other thing where Anthropic and OpenAI also discovered tens of thousands of more incidents, which they haven't even figured out what they are or how extensive the damage is. Right now, as I'm speaking, they're trying to figure that out. It's gotten out of hand. AI agents, AI models, anywhere and everywhere, especially the most recent frontier ones that haven't even been released, so you and I don't even have access to it, are running rampant. So with this precedence, with this environment, it makes sense that the labs need to start making moves to try and get a hold of this entire thing. On the screen here, I have the first post from this guy called Tomek. Now Tomek leads Chain of Thought Monitorability at OpenAI. So basically reading the internal thoughts of an AI model to understand what it's really thinking. And he goes, listen, last week, I was called into a meeting with OpenAI's head of safety. They told me that they no longer trusted me and a security guard then took my badge and walked me out of the building. And that's when he found out that two of his other colleagues were also fired. Now the reason Tomek was given was that he had leaked important information to an external third party called METR. METR is an external third party that evaluates models before they're released to the public. Their whole intent and purpose is to make sure that models are safe. If that name rings a bell, it's because Dario Mode announced two or three weeks ago in his Pacing the Frontier letter that he wants to partner with external parties to come inside Anthropic and evaluate their internal models. METR was the example that he used. So again, a very prominent independent third party. But that's not even the craziest part.