2028 AGI, New Architectures, Aligning Superhuman Models with Shane Legg, Deepmind Founder, on The Dwarkesh Podcast

**Erik Torenberg** (0:00)
Turpentine is a network of podcasts, newsletters, and more covering tech, business, and culture, all from the perspective of industry insiders and experts.
We're the network behind the show you're listening to right now.
At Turpentine, we're building the first media outlet for tech people by tech people. We have a slate of hit shows across a range of topics and industries, from AI with Cognitive Revolution, to Econ 102 with Noah Smith. Our other shows drive the conversation in tech with the most interesting thinkers, founders, and investors like Moment of Zen and my show Upstream. We're looking for industry leading hosts and shows along with sponsors. If you think that might be you or your company, email me at erikaturpentine.co. That's E-R-I-K at turpentine.co.

**Nathan Labenz** (0:46)
Hello and welcome back to The Cognitive Revolution. Today, we're concluding our holiday bonus content series with an episode from the Dwarkesh Podcast.
I assume most listeners of The Cognitive Revolution are at least familiar with Dwarkesh, as he's had a number of major AI interviews mixed in with his generally excellent feed over the course of the last year, including AI safety philanthropist Holden Karnofsky, former GitHub CEO Nat Friedman, OpenAI chief scientist Ilya Setskaver, the legendary Eliezer Yudkowsky, my once-upon-a-time New York City roommate Carl Schulman, true story, Anthropics CEO Dario Amadei, and AI safety pioneer Paul Christiano. Across all those interviews, I have really appreciated how Dwarkesh has asked some of the most fundamental, critical questions in the most plain-spoken way to some of the most influential people in the field.
And that skill will be on full display today as we present Dwarkesh's recent interview with Deepmind co-founder and chief AGI scientist Shane Legg. I chose to feature this conversation because I think it's one of the most candid, straightforward assessments of AI scenarios that you'll hear anywhere. And it just happens to be coming from someone who not only foresaw where we'd be today, but founded an organization in Deepmind that has delivered breakthrough after breakthrough along the way. Overall, the views that Shane presents in this conversation match up extremely well with both my understanding of current AI systems weaknesses and my expectations for how and how soon they are likely to be overcome. So when I hear him say that he sees relatively clear paths forward to addressing most of the shortcomings we see in existing models, I don't hear that as guessing. Particularly in light of the fact that Deepmind published one of the recent state-space model papers that I covered in the Mamba episode. That was the Block State Transformers paper, and it came out just a few weeks after this episode originally aired. Of course, Deepmind has tons of other projects underway internally as well.
Overall, it really does seem to me that we're headed for some form of AGI over the next few years. And while we're definitely not ready for it, I appreciate how much and for how long Shane Legg has been thinking about this. He is a signer of the Center for AI Safety's one-sentence extinction risk statement, and he's one of the big reasons that I think the AI game board is in remarkably good shape overall. I just hope he's right that the challenge of AI alignment gets easier in important ways as the models become more sophisticated. All that and more make this short conversation a great jumping off point for 2024, year two of AI's second era. So to help you better calibrate your timelines, here is Dwarkesh Patel with Deepmind Chief AGI Scientist, Shane Legg.

**Dwarkesh Patel** (3:31)
Okay, today I have the pleasure of interviewing Shane Legg, who is a founder and the Chief AGI Scientist of Google Deepmind. Shane, welcome to the podcast. Thank you.
So first question, how do we measure progress towards AGI concretely? So we have these loss numbers, and we can see how the loss improves from one model to another. But it's just a number. How do we interpret this? How do we see how much progress we're actually making?

**Shane Legg** (3:59)
That's a hard question actually.
AGI by its definition is about generality. So it's not about doing a specific thing. It's much easier to measure performance when you have a very specific thing in mind because you can construct a test around that. Well, maybe I should first explain what do I mean by AGI? Because there are a few different notions around. When I say AGI, I mean a machine that can do the sorts of cognitive things that people can typically do, possibly more.
But to be an AGI, that's the bar you need to meet. If we want to test whether we're meeting this threshold or we're getting close to this threshold, what we actually need then is a lot of different kinds of measurements and tests that spans the breadth of all the sorts of cognitive tasks that people can do, and then to have a sense of what is human performance on these sorts of tasks, and that then allows us to judge whether or not we're there. It's difficult because you'll never have a complete set of everything that people can do because it's such a large set. But I think that if you ever get to the point where you have a pretty good range of tests of all sorts of different things that people do, cognitive things people can do, and you have an AI system which can meet human performance and all those things, and with some effort you can't actually come up with new examples of cognitive tasks where the machine is below human performance, then at that point, it's conceptually possible that there is something that the machine can't do that people can do. But if you can't find it with some effort, I think it will be practical purposes, you now have an AGI.