Anthropic Head of Pretraining on Scaling Laws, Compute, and the Future of AI

**Ankit Gupta** (0:05)
Hey guys, I'm thrilled to be joined today by Nick Joseph, the head of pre-training at Anthropic. To give viewers a high level sense of what we'll be covering, we're going to start with the basics of what pre-training is, and then dig into how Nick thinks about strategy, data alignment, and infrastructure at Anthropic. And by the end, you'll hopefully have a sense for how progress in AI comes directly from advances in pre-training. I would love to talk a little bit about your backstory and how you got to this point. Where did you work before Anthropic, and what were your takeaways from those places?

**Nick Joseph** (0:29)
Yeah, so let's see, I was at Vicarious, and then at OpenAI before Anthropic. So Vicarious was originally an AGI lab, and when I joined, they were making a shift to product, particularly working on robotics products. And the thing I worked on was training computer vision models for their robotics products. It was my first job, so I think I just learned a ton about how to do machine learning models, how to write machine learning infrastructure.

**Ankit Gupta** (0:53)
And at the time, were you also thinking about a career as an academic? At the time, a lot of people doing AI work were in PhDs. That's kind of what I was thinking about before I started to do a company. How were you thinking about that in your headspace?

**Nick Joseph** (1:03)
Yeah. So, I mean, actually, we went a little bit. I think a lot of my thinking on this had come from an internship I did at Givewell, which is a non-profit that evaluates charities. And some people there being like, at some point, we might have AGI, it could be dangerous. We should worry about these risks. This could be a big impact on humanity. And I was not super convinced at the time and went down the economics route and was going to try to work on directly helping people in poverty. That didn't work out for various reasons and ended up being like, okay, I'll at least work on AI. Either the safety thing will turn out to be important, I'll work on that or it won't be and I'll just make cool things with AI that can probably help people in poverty more. I wasn't really coming at it from an academic standpoint. I was like, in fact, when I switched to that, it was part of the appeal was that I could immediately go do stuff in AI. Whereas if I want to work in economic policy, I'd have to wait, I don't know, six years to a PhD and start and it's a longer path.

**Ankit Gupta** (1:53)
And what did the state of AI safety work at that time even look like? Who were the people who were thinking about that kind of stuff? There were some folks at Vicarious thinking about this kind of thing, but it was fundamentally a robotics company. And so how were you thinking about that at the time?

**Nick Joseph** (2:05)
Yeah. So my sense was at the time, a lot of the AI safety discussion was kind of theoretical. Like the models weren't actually that good. They weren't really posing these dangers. So it was a lot more like philosophical. It was like, oh, at some point we might get AI that's really smarter than humans and should we wait this future concern? How should we compare that to near term things? And I think that was actually just a less compelling argument. I think it was an interesting one and it made you think a bit.

**Ankit Gupta** (2:29)
So next you went to OpenAI. What was OpenAI like at this time?

**Nick Joseph** (2:32)
Yeah. So I was on one of the safety teams and kind of worked on, and then working on code models actually. When I got there, the first thing I saw was, they'd fine-tuned GPT-3 to write some code. But I add, it was really good. I was like, okay, if you're worried about AI getting really powerful, writing its own code, that seems like it could self-improve, and how likely is that to happen? So I was doing a bunch of evaluations and studies of what contributed. Then after eight months, basically everyone I worked with, all the safety leads left, which invited me to go to Anthropic, and that was the reason I joined OpenAI, was because I cared about AI safety and wanted to work with them.
So then I went with them to join Anthropic pretty much right when it started.

**Ankit Gupta** (3:17)
With that, why don't we transition a bit? These days you run the pre-training team specifically at Anthropic. Obviously, you've been working on pre-training at Anthropic for quite a bit of time and I'm sure it's evolved over the years, what that even entails and looks like. Why don't we start by just talking a little bit about what pre-training is? Like how does it even fit into the way of thinking about how AI models are developed at a place like Anthropic, and what exactly do you guys do?