“Escaping Human Control” - Anthropic CEO WARNS AI Needs A GLOBAL FREEZE

**SPEAKER_1** (0:00)
Study and play.

**SPEAKER_2** (0:02)
Come together on a Windows 11 PC.

**SPEAKER_1** (0:04)
And for a limited time, college students get the best of both worlds. Get the Unreal College Deal, everything you need to study and play with select Windows 11 PCs. Eligible students get a year of Microsoft 365 Premium, and a year of Xbox Game Pass Ultimate with a custom color Xbox wireless controller. Learn more at windows.com/studentoffer.
While supplies last, ends June 30th, terms at aka.ms slash college PC.

**Patrick Bet-David** (0:30)
The story comes out on Wall Street Journal just today, if not yesterday, Tom, and Anthropic. Robert, if you want to pull this up. And the story is about them talking about, it may not be a bad idea to put a pause on AI. Wait, what? Yeah.
Anthropic urges global pause in AI development, flags self-improvement risk. The one trillion dollar startup warns artificial intelligence models are nearing capability to improve without human intervention. We want to go a little bit lower. And there's a part that they say about scientists.
A quick summary, Anthropic suggests that top AI considerate Shoshalabh Devang got an Anthropic recently concluded in fundraising that valued trillion dollars. David Sacks, a venture capitalist, accused Anthropic of a regulatory capture agenda to slow competitor AI advances. Rob, can you type in science? And you know what that means, David Sacks. What David is trying to say is, and I'll get to that, some scholars at the former chiefs of the meta performance API. No, can you type in science?
Keep going with the war to the next section. I'm sorry, what's on powerful? Keep going to resemble some of the future, near future, and particularly, so just as I said, go to the next one, Rob.
Maybe this is the one that Connor send it to me. I don't know if you know which one I'm talking about. Connor send me a text in this. Let me read this. Is that it? AI systems that can advance themselves, known as full recursive self-improvement, could have the potential for a great good for science and health care. They also pose great risks for humanity, according to a blog post written by Maria Favaro, leader of Anthropic Institute. So a part of David Sacks, Tom, if you can correct me on this, my interpretation of what David Sacks is saying of Anthropic's position is, oh, now that you're at a trillion, you have a big leap, now you want regulation. Now you want it. Because you got such a big market here, now you want to regulate, so you make it tougher for the smaller. This is the one I'm looking for. Yes, we read the science fiction and watch science fiction here as well. They're talking about it, Anthropic. So it's not lost on us, Clark responded. How do you maintain control over fleets of scientists that are much, much larger and much faster than ones you've had before? How do you compete against them? Okay. Sacks is saying if you regulate, the smaller guy can't come in, so the barrier to entry becomes tougher for them. And now that the bigger players, it all almost becomes like a nationalized AI leaders, right? It's like all the business goes to a handful of banks, all the other smaller banks, let's let them go out of business to make the call. That's kind of what David Sacks is arguing against. If I'm not mistaken, that let's not regulate yet. Let's let the smaller guys compete. Tom, what are your thoughts about the story?

**Tom** (3:17)
Well, I'm in the David Sacks camp because it's called RCI, Recursive Self-Improvement.
Recursive, just think of it this way.
You ask AI to build you a simple app. And then you say, keep testing and improving it. And the AI is going to say, to what standard? To what number? To what do you want me to do? Do you want me to keep testing and improving this until you have whatever? Now, that's good for drugs if you're doing chemical analysis, chemical impact analysis, and you say, test it, then test it, then test it, and change it. That's recursive self-improvement.
So recursive self-improvement was going to get here. You were going to have it building software, testing software, and then improve that based on the test, then do it again. Well, the parent model that's being used there, let's say Opus 4.8, is fine, and it's doing that. But it has the limit of Opus 4.8.
And recursive self-improvement is just part of AI. That's what AI is doing. When they say you get to that, well, we're kind of here. And I agree with it. Oh, wait a minute. You know, one person ran the four-minute mile, so no one else can wear sneakers.