Sholto Douglas & Trenton Bricken — How LLMs actually think

**Dwarkesh Patel** (0:00)
Okay, today, I have the pleasure to talk with two of my good friends, Sholto and Trenton. Sholto.

**Sholto Douglas** (0:07)
Trust me, stop.

**Dwarkesh Patel** (0:17)
How about I start it with my good friends?
Anyways, Sholto, Noam Brown, the guy who wrote the diplomacy paper, he said this about Sholto. He said, he's only been in the field for 1.5 years, but people in AI know that he was one of the most important people behind Gemini success. And Trenton, who's an anthropic, works on mechanistic interoperability, and it was widely reported that he has solved alignment.
So this will be a capabilities-only podcast, alignment is already solved, so no need to discuss further. Okay, so let's start by talking about context links. It seemed to be underhyped, given how important it seems to be that you can just put a million tokens into context. There's apparently some other news that got pushed to the front for some reason, but yeah, tell me about how you see the future of long context links and what it implies for these models.

**Sholto Douglas** (1:26)
Yeah, so I think it's really underhyped because until I started working on it, I didn't really appreciate how much of a step up in intelligence it was for the model to have the onboarding problem basically instantly solved. And you can see that a little bit in the perplexity graphs in the paper where just throwing millions of tokens' worth of context about a code base allows it to become dramatically better at predicting the next token in a way that you'd normally associate with huge increments in model scale. But you don't need that. All you need is a new context.
So underhyped and buried by some other news.

**Dwarkesh Patel** (1:56)
In context, are they as sample efficient and smart as humans?

**Sholto Douglas** (2:01)
I think that's really worth exploring. For example, one of the evals that we did in the paper has it learning a language in context better than a human expert could learn that new language over the course of a couple months. This is only a pretty small demonstration, but I'd be really interested to see things like Atari games or something like that, where you throw in a couple hundred or a thousand frames, labeled actions, in the same way that you'd show your friend how to play a game and see if it's able to reason through.
At the moment, with the infrastructure and stuff, it's still a little bit slower doing that, but I would guess that might just work out of the box in a way that would be pretty mind blowing.

**Dwarkesh Patel** (2:37)
And crucially, I think this language was esoteric enough that it wasn't in the training day.

**Sholto Douglas** (2:41)
Right, exactly. If you look at the model before it has that context thrown in, it doesn't know the language at all, and it can't get any translations.

**Dwarkesh Patel** (2:48)
And this is like an actual human language.

**Sholto Douglas** (2:50)
Yeah, exactly, an actual human language.

**Dwarkesh Patel** (2:52)
So if this is true, it seems to me that these models are already in an important sense superhuman. Not in the sense that they're smarter than us, but I can keep a million tokens in my context when I'm trying to solve a problem, remembering and integrating all the information in an entire code base.
Am I wrong in thinking this is like a huge unlock?

**Sholto Douglas** (3:13)
I actually genuinely think that's true. Like previously, I've been frustrated when models aren't as smart, like you ask them a question and you want it to be smarter than you or to know things that you don't. And this allows them to know things that you don't in a way that just ingests a huge amount of information in a way you just can't.
So yeah, it's extremely important.

**Dwarkesh Patel** (3:31)
Well, how do we explain in-context learning?

**Sholto Douglas** (3:34)
Yeah, so there's a piece of, there's a line of work I quite like where it looks at in-context learning as basically very similar to gradient descent, but the attention operation can be viewed as gradient descent on the in-context data. That paper had some cool plots, so it basically showed we take n steps of gradient descent and that looks like n layers of in-context learning. It looks very similar. So I think that's one way of viewing it and trying to understand what's going on.

**Dwarkesh Patel** (3:58)
Yeah, and you can ignore what I'm about to say because given the introduction, alignment is solved and our safety isn't a problem, but I think the context stuff does get problematic, but also interesting here. I think there'll be more work coming out in the not too distant future around what happens if you give a hundred shot prompt for jail breaks, adversarial attacks. It's also interesting in the sense of if your model is doing gradient descent and learning on the fly, even if it's been trained to be harmless, you're dealing with a totally new model in a way. You're fine-tuning, but in a way where you can't control what's going on. Can you explain what do you mean by gradient descent is happening in the forward pass and attention?