Zvi Mowshowitz on Longer Timelines, RL-induced Doom, and Why China is Refusing H20s

**Erik Torenberg** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, for a record 10th time, my friend Zvi Mowshowitz returns for another wide-ranging conversation about the state of AI as we head in to the final months of 2025 I assume that Zvi needs no introduction, but for anyone who's somehow not already familiar with his work, he writes the essential blog, Don't Worry About the Vase, where he chronicles AI developments and offers a breadth and depth of analysis you won't find anywhere else.
We begin today with a discussion of why so many of the sharpest minds in the AI space, including Zvi, are now projecting somewhat longer timelines to AGI, despite the fact that GPT-5 is now generally understood to be right on trend, and that multiple model developers achieved an IMO gold medal this summer, which would have seemed miraculous just a few years ago. He also explains why his PDoom is, if anything, a bit higher than last time we talked, highlighting that the US government seems increasingly captured by commercial interests, and also that reinforcement learning appears to make models less aligned in fundamental ways. This leads into a fascinating discussion of why CLOD 3, Opus, still seems to have been uniquely durably aligned, why we should believe that the training techniques that imbued later CLOD models with higher levels of agency come with their own alignment compromises, and why we should never attempt to supplement outcome-focused reinforcement learning with techniques meant to suppress unwanted behaviors in a model's chain of thought or internal states. Namely, that while such techniques can make things better in the immediate term, they risk teaching models to hide their true reasoning while still incentivizing the same bad behavior. Beyond that, I of course had to get Zvi's latest Live Players Analysis, including whether any Chinese companies remain Live Players in the wake of Beijing's decision to refuse H20s, whether there's any rational basis for that decision, and also whether or not XAI might derive a special advantage from access to the steady stream of challenging problems that engineers are solving at Elon's other hard tech companies. Toward the end, we compare notes from our parallel experiences as recommenders for the AI Safety Focused Survival and Flourishing Fund. With the main conclusion being that the AI safety sector has many more worthy projects than current resources can support, which means that the world could benefit tremendously from new donors getting involved. If you needed one more reason to subscribe to Zvi's blog, he's planning a more comprehensive write up of AI Safety Giving Opportunities later this year. Finally, as always, I ask Zvi what he sees as especially virtuous to do now. And since Say What You Really Think was a big part of the answer, I will note that while Zvi and I tend to agree much more often than not, there were a few moments in this conversation, particularly when it comes to the US decision to go ahead and sell H20s to China, where I either don't agree with his conclusions or am at the very least extremely uncertain. Nevertheless, considering that we were already running super long and that regular listeners have heard my takes on other episodes, I didn't bother to debate those points on this particular occasion. With that, I hope you enjoy this opportunity to pick one of the brightest and most informed human minds in the AI space. This is the great Zvi Mowshowitz. Zvi Mowshowitz, welcome back to The Cognitive Revolution.

**Zvi Mowshowitz** (3:23)
Thanks. Great to be here.

**Erik Torenberg** (3:25)
Always exciting. Let's start with timelines today. I am a little confused about the way in which people seem to be updating over the course of this summer of 2025 We've had GPT-5, obviously, where I would say it's safe to say the launch was not exactly super smooth.
And people were a little disillusioned with that at first. Now the dust has settled and it seems like people have kind of mostly come around to, it is actually a good model and it's basically on trend. It's actually still a little bit above the all-important meter task length curve.

**Zvi Mowshowitz** (4:04)
It's on a very shift upward of that graph.

**Erik Torenberg** (4:08)
The four-month doubling instead of the seven-month doubling.
And also something that seemed really important to me that happened this summer is, of course, we got IMO Gold. And we got a very close to number one finish in the world, although it ended up number two on competitive programming competition. And yet, with all these things, people, including like really smart people, I'm not talking about denialists here, but like Ryan Greenblatt and Daniel Pocatello type people who are as plugged in as you get. And I would say smarter than me, you can evaluate for yourself. Timelines seem to be lengthening. So have you updated your timelines, if at all? And what do you make of the seemingly on-trend or even maybe a little ahead of schedule events that are still cashing out to people as longer timelines overall?