Ryan Greenblatt – What happens once AI can automate AI research?

**Dwarkesh Patel** (0:00)
Today, I'm chatting with Ryan Greenblatt, who is the Chief Scientist at Redwood Research, where he focuses on technical AI safety and security work. I want to talk to you about recursive self-improvement. This is the idea that once you build human level intelligences, they quickly slingshot towards tens of billions of super intelligences, which are each individually more competent than the top human experts across every field.
Whether or not this turns out to be the case, I think is actually probably the most important question in the world right now. Historically, I've been quite skeptical that this kind of thing happens, but you seem to think that it might be plausible and so I wanted to hear the case for it.

**Ryan Greenblatt** (0:36)
Yeah, let's talk about this. So first, I think it's worth noting that R&D is a type of task at which the AIs are especially good, because both the companies are trying really hard to make their AIs good at R&D, and it's the kind of domain. It has a lot of nice properties from the perspective of how AI development works right now, so it's pretty verifiable. You can do a bunch of stuff iteratively and he'll climb on various metrics. Then I think once you have AIs which are roughly matching the top human experts in R&D, that could kick off a feedback loop where the AIs are doing AI research, that puts the smarter AIs, that feeds back in, and that feedback loop could be strong enough that you end up with a lot of progress in a short period of time. Maybe my median expectation is something like four or five years of AI progress in a single year. This requires really overcoming a huge amount of diminishing returns in research, and basically doing the equivalent of what progress we would have gotten after a really large compute scale out. So this is a pretty impressive big thing. It's worth keeping in mind that five years of AI progress, four years of AI progress, even three years of AI progress, is really a lot of fucking high progress. So right now, it's like three years ago, or a little over three years ago, there was GPT-4 that had come out.
Right now, of course, we have like Mythos 5 or whatever and maybe a somewhat better model that Anthropic has internally.
And so, that is just a huge amount of progress in a bit over three years. And if we're talking about five years, then maybe we're talking more about like a jump from GPT-3 to Mythos 5 or whatever. Yeah.

**Dwarkesh Patel** (2:06)
Okay. So, I think this argument has three different parts and now I want to evaluate each one of them. First is the argument that AI R&D is very, very viable. Second is the argument that if you automate AI R&D, you could get four or five years of progress in a single year.
Third is the argument that what comes out the other end of four or five years of AI progress at the current pace, starting at the starting point to whenever AI R&D is automated.

**Ryan Greenblatt** (2:30)
Yeah.

**Dwarkesh Patel** (2:31)
What comes out the other end is an AI where you can drop it on the job at basically anything you can imagine. You can drop it in Texas politics in the 1940s and it out maneuvers Lyndon Johnson. You can drop it in, I don't know, TSMC and it does better process engineering at TSMC. It's certainly a better video editor than I.
My video editors are very excellent, but it is just in general better than humans at any given job that it finds itself trying to do. So, I want to evaluate all of these sub-arguments that lead to basically getting ASI pretty soon after this benchmark, which you're expecting by 2030 or something, right?

**Ryan Greenblatt** (3:10)
Yeah, I would say that I expect like full automation of AR&D perhaps somewhere around like 2031, 2030, and then getting to like the like beats all humans on the job milestone. Maybe I expect median around 2033, but sort of like if I see AIs fully automating AR&D, I think I'm expecting that probably within a year. It's just like the way the forecasting works out. There's a difference between medians is bigger than the median difference between milestones. Anyway, whatever.

**Dwarkesh Patel** (3:36)
By the way, there's this meme on the Internet because every time I'm trying to ask about people's timelines, I'm asking Dario or somebody, I'm always like, okay, how long before you can automate my video editors?
There's this meme of my video editor editing the podcast. Every time I listen to this. The reason I do it is because I think it's easy to get lost in abstractions when you talk about jobs you don't understand well, and to very concretely understand what it takes to automate a job that I actually understand why it's difficult for LLMs to currently take control over.