Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary

**Dwarkesh Patel** (0:00)
So today, I'm interviewing Gwern Branwen. Gwern is an anonymous internet researcher and writer. He's deeply influenced the people who are building AGI. He was one of the first people to see LLM scaling coming. If you read his blog, you know he's one of the most interesting polymathic thinkers alive. We recorded this conversation in person. In order to protect Gwern's anonymity, we created this avatar. This isn't his voice, this isn't his face, but these are his words. Gwern, what is the most underrated benefit of anonymity?

**Gwern Branwen** (0:33)
I think the most underrated benefit of anonymity is that people don't project on to you as much. They kind of can't like slot you into any particular niche or identity and like end up writing you off in advance. You know, everyone has to read you at least a little bit to even begin to dismiss you. It's great that people can't retaliate against you. And I've derived a lot of benefit from people not being able to like mail heroin to my home and call the police to swap me. But I always feel that the biggest benefit is just that you get a hearing at all basically. You don't get immediately written off by the context.

**Dwarkesh Patel** (1:09)
Do you expect companies to get automated top down starting with the CEO or from the bottom up starting with the workers?

**Gwern Branwen** (1:17)
All the pressures I think are to go bottom up. And from existing things, it's just much more palatable in every way to start at the bottom and replace there and then work your way up to eventually kind of just having human executives overseeing a firm of AIs. And also from an RL perspective, I think if we are in fact better than AIs in some way, it should be in the long term vision thing, right? Like the AIs will be too myopic to execute any kind of novel long term strategy and seize new opportunities.
So that would presumably give you this paradigm where you have like a human CEO who does the vision thing and then the AI corporation kind of like scurries around underneath them doing the CEO's bidding. And they don't have the taste that the CEO has. So you have one kind of Steve Jobs figure at the helm and then maybe a whole pyramid of AIs out there executing the vision and bringing him new proposals. And he looks at every individual thing and says, no, like that proposal is bad. This one is good. That may be hard to quantify, but I think that human led firms should, under this view, end up outcompeting the entirely AI firms, which would keep making these myopic choices that just don't quite work out in the long term.

**Dwarkesh Patel** (2:37)
What is the last thing that you think you personally will be doing before your last keystroke is automated?

**Gwern Branwen** (2:43)
The last thing that I see myself still doing right before the nanobots start eating me from the bottom up and I start screaming, no, I specifically requested the opposite of this, is I think right before that, I think what I'm still doing is the Steve Jobs kind of thing of choosing. So my AI minions are bringing me wonderful essays. I'm saying this one is better. This is the one that I like and possibly building on that and saying, that's almost right, but you know what would make it really good if you pushed it to 11 and this way.

**Dwarkesh Patel** (3:15)
If you do have firms that are made up of AIs, what do you expect the unit of selection to be? Will it be individual models? Will it be the firm as a whole? I mean, with humans, we have these debates about whether it's kin level selection, individual level selection, gene level selection. What will it be for the AIs?

**Gwern Branwen** (3:32)
Yeah, I think once you can replicate individual models perfectly, the unit of selection can move way up and you can do much larger groups and packages of minds. That would be an obvious place to start. You can train individual minds in a differentiable fashion, but then you can't really train the interaction between them, right? So you'll have groups of models or minds of people who just work together really well in a global sense, even if you can't attribute it to any particular aspect of their interactions. There's some places you go and people just like work really well together, and there's nothing specific about it, but for whatever reason, they all just click in just the right way. So I think that seems like the most obvious unit of selection. You would have like packages, I guess possibly like department units, where you have a programmer and a manager type, then you have maybe a secretary type, maybe a financial type, a legal type. This is the default package where you just copy everywhere you need a new unit. And at this level, you can start evolving them and making random variations to each of the packages, and then keep the one that performs best.