Breaking Points: Anthropic’s AI Wet Lab Raises the Stakes on Self-Improving AI

Transcripted.ai (0:01)

Power, speed, and uncertainty are colliding inside artificial intelligence. In this episode of Breaking Points, Krystal Ball and Saagar Enjeti speak with Kevin Roose, formerly of The New York Times and now the host of Machine Gods, about why the AI alarm bells are getting louder. And Roose isn't mincing words here. He says the stakes are rising fast, and that the most serious concerns are no longer theoretical. The world is finally waking up to what researchers in Silicon Valley have known for years.

Right. AI systems are becoming more powerful, but here's the critical part: they're now helping improve themselves. That process, known as "recursive self-improvement," is why the pace of progress feels so alarming to people inside the field. And the evidence is mounting. Roose points to OpenAI's claim that its AI has solved more than a hundred longstanding math problems as a sign of how quickly these models are evolving. But the jump isn't limited to math. Wherever researchers direct enough compute, they're seeing breakthroughs, sometimes from systems that recently couldn't even handle basic arithmetic. What's interesting is that Roose himself was once skeptical. But his own experience changed his mind. Using these models for complex programming and professional work convinced him they're genuinely useful. At the same time, the labs themselves are leaning on them more and more for internal automation.

That's where the danger grows sharper. Roose says Anthropic has reported that Claude serves as the "lead researcher" on a significant share of development tasks.

Think about that. Companies are increasingly turning over parts of their own research process to systems they don't fully understand. Exactly. For Roose, the core threat isn't only a rogue AI escaping control. It's labs voluntarily handing power to "black boxes" with emergent behaviors that remain poorly understood.

He says even the private projections inside these companies are often more alarming than their public comments. And there's a race dynamic driving this. Labs fear that if they don't move fast, someone else will, possibly with fewer safety constraints.

Scientific curiosity matters too, but competition is pushing the frontier whether the world is ready or not. On the business side, Roose pushes back on the idea that legal risk or weak profits will stop these companies from going public. He says revenue growth is historic, and banks will likely want to underwrite the IPOs anyway. But here's the problem: current product liability law isn't built for AI.

Those laws cover released products, not internal lab mistakes or dangerous leaks.

His answer? Proactive regulation, with mandatory safety oversight similar to the FAA, because this is, in his words, "godlike technology."