Final Analysis on CA’s AI Bill SB 1047 with Nathan Calvin, Dean W. Ball, and Steve Newman

**SPEAKER_1** (0:02)
Hey, everyone. I'm excited to tell you about our newest show on the Turpentine Network called 1 to 100 1 to 100 is a weekly deep dive with founders of notable hyper growth companies that could be your next career move. On the upcoming season, you'll get the inside scoop on breakout companies like Modal Labs, Clay, and Mercor. You'll hear directly from the founders about how they built their products, shaped their company cultures, and scaled their businesses from 1 to 100 and beyond. If you're considering a career change or just want to stay ahead of the curve in tech, 1 to 100 will give you an edge. A link is in the description. Listen wherever you get your podcasts.

**Nathan Labenz** (0:40)
Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. This episode is brought to you by WorkOS. If you're building a B2B SaaS application, at some point your customers will start asking for enterprise features like SAML authentication, SCIM provisioning, role-based access control, and audit trails. That's where WorkOS comes in. With easy to use and flexible APIs that help you ship enterprise features on day one without slowing down your core product development. Today, some of the hottest startups in the world are already powered by WorkOS, including ones you probably know like Proplexity, Vercel, Jasper, and Webflow. WorkOS also provides a generous free tier of up to one million monthly active users for user management, making it the perfect authentication and authorization solution for growing companies. It comes standard with rich features like bot protection, MFA, roles and permissions, and more. If you're currently looking to build SSO for your first enterprise customer, you should consider using WorkOS. Integrate in minutes and start shipping enterprise plans today.
Hello, and welcome back to The Cognitive Revolution. Today, we're revisiting California's AI Build SB 1047, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act. If you haven't been following closely, we first covered this topic back in May. At that time, the bill required Frontier Model Developers to provide reasonable assurance that their models would not contribute to causing large-scale harms. And it was designed in such a way that it would have applied to a greater and greater percentage of AI development over time. I was inclined to support it, as it seemed to be a genuinely good faith effort to focus the burden on Frontier players in a way that likely wouldn't have much slowed practical adoption today. Well, a lot has changed since then, and we're here to unpack the latest developments. I'm joined once again by Nathan Calvin from the Center for AI Safety Action Fund, one of the bill's co-sponsors, Dean W. Ball from the Mercatus Center, who's become a prominent critic of the bill, and Steve Newman, who's working to elevate the discourse around AI issues generally. Their perspectives have evolved along with the bill, and I think you'll find our discussion both nuanced and hopefully illuminating. The updated version of SB 1047 is less ambitious, but in my view, probably better overall. It now places less emphasis on concrete implementations of safeguards, applies only to models which cost $100 million or more to develop, and has the primary effect of requiring frontier companies to develop and publish safety plans and protocols. Given where we already are with AI development, I believe that forcing the companies who are developing never-before-seen AI capabilities to disclose their safety plans and protocols is simply a prudent step. With so many leading AI luminaries sounding the alarm, all surveys of broader groups of AI researchers showing extreme uncertainty, and all popular opinion polls and plain common sense supporting more investment in safety and oversight for developers, it's really not too much for the state of California to ask. With the final vote likely this week and the governor's decision not far behind, I will say it plainly. If I were the governor, I would sign the bill. To be clear, that's not a stance I've taken lately. I never thought the bill would be the crazy chill on AI research that some had claimed. But there was a point when certain revisions seemed to grant what seemed to me to be too much power to the now entirely removed Frontier Model Board. And at that point, I was inclined to withdraw my tentative support, as a misguided board could easily have done more harm than good. The legislative process, however, despite some counterproductive hyperbole and vitriol, which you've probably seen online, has ultimately, seemingly, landed in a pretty good place. In the final analysis, as a friend of the show, Zvi Maschowitz aptly puts it, this has primarily become a transparency bill. Anthropix Open Letter, which everyone seems to be reading as a tacit endorsement, offers solid, plain-spoken analysis. Things are moving extremely fast. We need to create incentives to identify and address extreme risks. Being honest with the public is good. And while there are no perfect solutions at this point, the pros probably outweigh the cons. On the other hand, OpenAI's letter, opposing the bill and calling instead for clarity and certainty for AI developers from federal AI regulations, given the massive confusion they've directly caused in so many ways over the last year, including the recent Strawberry Emoji Hype Fest, really does not move me at all. And I say this as someone who is super hyped myself about the future of OpenAI's products, including their advanced voice mode, which I just started using this week. But that's a topic for another day. If I have one remaining criticism of the current bill, it would be that the $10 million threshold at which responsibility shifts from the original model developer to the fine tuner seems too high. That is extremely rare air for a fine tuning budget and will apply to a very small number of fine tuning projects. This seems both potentially unfair to Meta and seems to let everyday application developers off a bit too easy. I don't think that we want to require lengthy published policies from small application teams. But I do think that application developers should share in responsibility, especially to the degree that they are selling autonomous or agent style products that can take real world actions. For now, I want to extend my appreciation to Nathan Calvin, Dan Hendricks, the Center for AI Safety, and Senator Scott Weiner for their hard work and openness to suggestions for improvement. Everything I've seen suggests that this is a Herculean labor of public service for Senator Weiner, and that he's been trying really hard to find the right compromise that balances all interests. I also appreciate Dean for recognizing how much better the bill is now as we reach the end of the process than it was in the beginning. He and other critics of the bill deserve credit for the positive changes too, even if they still don't support the final product. Finally, it's worth noting for those who may know Dan Hendricks only from these policy debates that he is actually a world-class researcher. His recent work on tamper-resistant safeguards for open-weight language models, while still early in development, holds the promise of making it possible for a company like Meta to remove harmful behavior in a way that downstream users could not easily undo, which is unfortunately the case for today's safety techniques. I hope to have Dan on for a technical research-oriented episode covering this and other topics soon. As always, if you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends. I always welcome your messages either via our website, cognitiverevolution.ai, or you can DM me on any social network. Now, let's review the final version of California's AI Bill SB 1047 with Nathan Calvin, Dean W. Ball, and Steve Newman. Nathan Calvin from the Center for AI Safety, Dean Ball from the RECADA Center, and Steve Newman from an AI Sense-Making Project, which might soon have an official name. Welcome back to The Cognitive Revolution.