Ep 164: Inside Anthropic with Sholto Douglas & Nicholas Marwell — AGI, Optimism vs Risks, Regulatory Capture, Open Source & More!

Sholto Douglas (0:00)

Models which are as or more capable than all humans are very likely to occur in the next couple of years.

Nicholas Marwell (0:06)

Each marginal unit of intelligence is worth exponentially more than the unit that came before.

Sholto Douglas (0:11)

You can keep increasing and potentially doubling the productive intellectual and physical capacity of humanity in such a way that allows you to truly get to a post-scarcity world.

Joe Lonsdale (0:20)

You guys are the edge of the AI frontier. What are the most exciting things you're working on?

Nicholas Marwell (0:23)

AI is going to be the most important technology in the life sciences certainly of our lifetime and maybe ever.

Sholto Douglas (0:29)

But it also comes with massive risks.

Nicholas Marwell (0:31)

We shouldn't be releasing these technologies into the world before we can keep the bad actors from doing a lot of bad things.

Joe Lonsdale (0:37)

Tech leaders I know who are more skeptical, who are saying, well, Dario is trying to scare everyone to create more rules to slow things down.

Sholto Douglas (0:43)

So far, we've slowed ourselves down much more than anyone else.

Joe Lonsdale (0:48)

Is there any scenario you'd just slow America and let China go ahead?

Nick Marwell and Sholto Douglas are key technical leaders at Anthropic, the fastest growing AI company in the world. They're both very optimistic about the future despite some really serious threats we're facing as a civilization. What are these threats? How do we think about offensive versus defensive dynamics in bio, in cyber, in other key areas? Some people are pretty angry at Anthropic for regulatory capture. What is their view on this? What is Anthropic trying to do? What do they think of the battles with distillation in China? If everyone's able to copy their models, how is Anthropic still going to be worth trillions of dollars and why? If you're a young person in the workforce, if you're trying to build your career but you're not in the center of AI, what should you be doing for the next 10 years? An optimistic insider's view on one of the fastest moving companies in the world at a fascinating time. Excited for you to hear from them. Welcome to American Optimist. Really excited to have today two amazing talents from Anthropic. Nick, you're the key member of the technical staff, help lead RL Science, reinforcement learning science, and Sholto Douglas, also your tech lead in reinforcement learning, is that right? Yep.

And how long have you guys been in Anthropic?

Nicholas Marwell (2:02)

Just over three years for me.

Joe Lonsdale (2:03)

The company's only, how old though? It's five years or so. Yeah, so it's amazing. It's like one of the most important, biggest companies, fastest growing companies now in the world, and it's only been around for five years. So this is a heady time here in California, filming here in Napa today. Thanks for being here, guys.

Sholto Douglas (2:17)

Thank you for inviting us. It's going to be fun.

Nicholas Marwell (2:18)

Thanks for having us.

Joe Lonsdale (2:19)

Let's start with your backgrounds. So where did you grow up, Sholto? Where are you from?

Sholto Douglas (2:23)

I grew up in Sydney. I got very convinced that scaling and in general, we were on track to get AGI in the 2020s, in about 2020, reading a mixture of blog posts and papers.

And I worked very hard on nights and weekends to prove that I could do the work with a quality bar that was demanded by DeepMind and Anthropic and OpenAI and these kind of companies.

Joe Lonsdale (2:46)

How did you study computer science?

Sholto Douglas (2:48)

I studied robotics in undergrad. So I graduated in 2019 and then I was doing my own research basically.

Ended up working for DeepMind for two years and at Anthropic for the last 18 months.

Joe Lonsdale (3:00)

Awesome. Nick, where are you from?

Nicholas Marwell (3:02)

Yeah, I'm actually from San Francisco originally, which is rare nowadays, I suppose, in folks in AI. I grew up in a household with a father in particular, who had spent the first half of his career doing very traditional business things, and then had found this interesting path in the latter half of his life and my more formative years, where he had gone and spent his time trying to figure out how to give back, and in particular, figuring out how to leverage technology to give back.

And I think a lot of, as I was thinking about what I wanted to do, I was looking for things that let me get this experience of both at the same time building the skills to become an effective capitalist, or whatever you want to call it, and also to use this to be able to make social good. I think Anthropic was this very perfect coincidence, where I was getting very concerned about AI safety. I was actually a resident working on a very different company at Thrive Capital at the time, and it was just one of these things where it came out of nowhere, and I was like, I think this is the thing for me to go do.