Anthropic's Responsible Scaling Policy, with Nick Joseph, from the 80,000 Hours Podcast

**Nathan Labenz** (0:00)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. This episode is brought to you by WorkOS. If you're building a B2B SaaS application, at some point, your customers will start asking for enterprise features like SAML authentication, skim provisioning, role-based access control, and audit trails. That's where WorkOS comes in, with easy to use and flexible APIs that help you ship enterprise features on day one without slowing down your core product development. Today, some of the hottest startups in the world are already powered by WorkOS, including ones you probably know, like Perplexity, Vercel, Jasper, and Webflow. WorkOS also provides a generous free tier of up to one million monthly active users for user management, making it the perfect authentication and authorization solution for growing companies. It comes standard with rich features like bot protection, MFA, roles and permissions, and more. If you're currently looking to build SSO for your first enterprise customer, you should consider using WorkOS. Integrate in minutes and start shipping enterprise plans today.
Hello, and welcome back to The Cognitive Revolution. Today, we're sharing a cross post from the 80,000 Hours podcast, in which host Rob Wiblin interviews Nick Joseph, head of model pre-training at Anthropic, about the responsible scaling policy that governs the company's frontier model development. I've been following the development of these policies and other voluntary commitments from Top Labs for much of the last year. At one point, I had the opportunity to participate in a workshop with Anthropic team members in which we reviewed, discussed, and offered comment on the responsible scaling policy draft. However, I could not easily convert that experience to a podcast, and so I was particularly excited to see this episode published, and really appreciate that the 80,000 Hours podcast team has allowed me to repost it here. On the substance of the matter, I of course very much appreciate how hard Anthropic is thinking about AI risks and what can be done about them, and also how transparent and even candid they are willing to be about the substantial uncertainty that remains. I'm also very glad that their example seems to have inspired others. OpenAI and Google have since published similar policies, and I understand that XAI is working on one as well. Unfortunately, however, the trend does not yet appear to be universal. Meta, for example, to the best of my knowledge, has not published a policy describing how it plans to evaluate models during training, let alone how they plan to proceed if it turns out that they are developing dangerous capabilities. As we happen to be sharing this episode with just a few days left before California Governor Kevin Newsom will have to sign or veto SB 1047, I will take a moment to say one more time that while it may not be the perfect AI safety bill, and indeed the release of OpenAI's O1 model, and the emerging paradigm of scalable inference time compute generally, do suggest that the definitions in the bill would need to be updated to stay relevant over time. In my view, the public does deserve to know what Frontier Labs are doing, and that goes double for Meta and any other companies who are openly sharing model weights. So whatever the fate of SB 1047 may be, I do think that we will need some measure that forces Frontier model developers to publish detailed safety plans for public scrutiny. In the last hour of this episode, Rob and Nick changed topics to focus on AI safety career advice. While this is not a sponsored episode, Nick's experience with 80,000 Hours Career Advising Service echoes my own, and presents a natural opportunity to remind you that 80,000 Hours is now offering free, one-on-one career advising sessions to Cognitive Revolution listeners. I encourage everyone to sign up for a free session at 80,000hours.org/cognitiverevolution, and especially so if you are one of the experienced software engineers that Nick notes are in such high demand among AI companies right now. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it online with friends, or write us a review on Apple Podcasts or Spotify. Your feedback is always welcome too, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. Now, for an in-depth conversation about Anthropic's responsible scaling policy and breaking into AI safety in your career, here's Nick Joseph from Anthropic with host Rob Wiblin of the 80,000 Hours Podcast.