Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold

**Demis Hassabis** (0:00)
I wouldn't be surprised if we had AGI-like systems within the next decade. It was pretty surprising to almost everyone, including the people who first worked on the scaling hypotheses, that how far it's gone. In a way, I look at the large models today, and I think they're almost unreasonably effective for what they are. It's an empirical question whether that will hit an asymptote or a brick wall. I think no one knows.

**Dwarkesh Patel** (0:21)
When you think about superhuman intelligence, is it still controlled by a private company?

**Demis Hassabis** (0:25)
As Gemini becoming more multimodal, and we start ingesting audiovisual data as well as text data, I do think our systems are going to start to understand the physics of the real world better. The world is about to become very exciting, I think, in the next few years as we start getting used to the idea of what true multimodality means.

**Dwarkesh Patel** (0:44)
Okay, today it is a true honor to speak with Demis Hassabis, who is the CEO of DeepMind. Demis, welcome to the podcast.

**Demis Hassabis** (0:51)
Thanks for having me.

**Dwarkesh Patel** (0:52)
First question, given your neuroscience background, how do you think about intelligence? Specifically, do you think it's like one higher level general reasoning circuit, or do you think it's thousands of independent subskills and heuristics?

**Demis Hassabis** (1:05)
Well, it's interesting because intelligence is so broad and what we use it for is so generally applicable. I think that suggests that there must be some sort of high level common things in a common kind of algorithmic themes, I think, around how the brain processes the world around us. So of course, then there are specialized parts of the brain that do specific things. But I think there are probably some underlying principles that underpin all of that.

**Dwarkesh Patel** (1:37)
Yeah. How do you make sense of the fact that in these LLMs though, when you give them a lot of data in any specific domain, they tend to get asymmetrically better in that domain? Wouldn't we expect a sort of general improvement across all the different areas?

**Demis Hassabis** (1:51)
Well, I think you, first of all, I think you do actually sometimes get surprising improvement in other domains when you improve in a specific domain. So for example, when these large models sort of improve at coding, that can actually improve their general reasoning. So there is some evidence of some transfer, although I think we would like a lot more evidence of that. But also, that's how the human brain learns too, is if we experience and practice a lot of things like chess or writing, creative writing or whatever that is, we also tend to specialize and get better at that specific thing, even though we're using sort of general learning techniques and general learning systems in order to get good at that domain.

**Dwarkesh Patel** (2:31)
Yeah. Well, what's been the most surprising example of this kind of transfer for you? Like you see language and code or images and text.

**Demis Hassabis** (2:37)
Yeah. I think probably, I mean, I'm hoping we're going to see a lot more of this kind of transfer, but I think things like getting better at coding and math, then generally improving your reasoning. That is how it works with us as human learners, but I think it's interesting seeing that in these artificial systems.

**Dwarkesh Patel** (2:55)
Can you see the sort of mechanistic way in which, let's say in the language and code example, there's like, I found the place in a neural network that's getting better with both the language and the code, or is it that too far down the weeds?

**Demis Hassabis** (3:06)
Yeah, well, I don't think our analysis techniques are quite sophisticated enough to be able to hone in on that. I think that's actually one of the areas that a lot more research needs to be done on kind of mechanistic analysis of the representations that these systems build up. And, you know, I sometimes like to call it virtual brain analytics in a way. It's a bit like doing fMRI or single cell recording from a real brain.
What's the analogous sort of analysis techniques for these artificial minds? And there's a lot of great work going on on this sort of stuff. People like Chris Ola. I really like his work and a lot of computational neuroscience techniques, I think, could be brought to bear on analyzing these current systems we're building. In fact, I try to encourage a lot of my computational neuroscience friends to start thinking in that direction and applying their know-how to the large models. Yeah.

**Dwarkesh Patel** (3:59)