OpenAI's Identity Crisis: History, Culture & Non-Profit Control with ex-employee Steven Adler

**Erik Torenberg** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, my guest is Steven Adler, former research scientist at OpenAI, author of a new substack, stevenadler.substack.com on how to make AI go better, and one of the 12 former OpenAI employees who recently filed an amicus brief to the Elon Musk versus OpenAI lawsuit, arguing that OpenAI's non-profit status and mission has been central to its historical success and that it should remain under non-profit control going forward. Of course, you probably know that there's been a major development in the OpenAI story this week. On Monday, OpenAI announced that it's changing plans and now intends to form a new public benefit corporation which will remain under non-profit control. While this news would seem to resolve the question that spurred this episode, the conversation itself remains highly relevant. As we spoke very little about the details of the case itself and much more about OpenAI's history, the evolution of its company culture, and the prevailing values, attitudes, and mindsets at the company today. To begin, Steven takes us back to his early days at OpenAI. He joined shortly after the original GPT-3 API was launched, and he recounts a pivotal moment in the company's history, the departure of important research and other leadership to found Amtropic, and how the effort that OpenAI leadership went to reaffirm its non-profit status and its commitment to its mission was central to keeping the company together through that crisis.
We then explore the four chapters of Steven's tenure at OpenAI, including his work on product safety, including the GPT-4 deployment, on dangerous capability valuations, on proof of personhood techniques and related plans for identifying and authorizing AI agents, and finally on AGI readiness. From there, we get Steven's perspective on OpenAI's evolution from a research-focused organization to a hyper-growth technology company. We discuss Steven's understanding of OpenAI leadership's motivations, their relationship to AI safety concerns, and the ways in which their commitments to safety testing have eroded over time, their attitude toward the possibility of recursive self-improvement, the contrasting cultural forces within the company, and more. Overall, I found Steven to be very level-headed and even-handed. At times, I'd even say charitable. Which does provide some valuable context for the reactions to this week's news that we've seen from the Amici and other OpenAI watchers. Personally, when I first read the news that the non-profit will retain control, while also owning enough stock to fund many worthy philanthropic projects, it seemed to me a clear win for Steven and friends. Steven hasn't commented since the news, but the general reaction online, I would describe as ranging from cautious optimism to outright cynicism. That they'd want to see and really have a chance to scrutinize the details of such an arrangement is obviously prudent, but the evident suspicion that OpenAI may be playing word games or otherwise trying to trick the public kind of surprised me. If nothing else reflects just how low trust has fallen between these former team members and OpenAI leadership. So where does all this leave us? As someone who's watched OpenAI closely but never worked directly with anyone on the leadership team, I can really only speculate. But here are two things that seem likely true and important, at least to me. First, as Sam has indicated multiple times, OpenAI is making all of this up as they go along. They have no precedent to guide them and no choice but to keep moving forward. Considering everything that he and the executive team are juggling, from developing and productizing transformative technology, to managing historic fundraisers, internal ideological divides, high-profile departures, PR crises, potential regulation, and of course corporate restructuring, brute force time constraints mean that Sam is probably spending less time on many of these critical issues than many outside analysts. Obviously, this isn't ideal. But it's also not inconsistent with the idea that they may really be sincerely motivated and genuinely trying their best to ensure that AI benefits all humanity. Second, regardless of their governance structure, there is huge value in the work that these outside analysts, commenters, ex-employees, and government officials are doing to help steer the company in the right direction. They are making no secret of their ambition to transform life as we know it. And it remains strikingly plausible that this one company could play a pivotal role as we enter a future in which AI utopia, dystopia, or even outright human extinction are all live possibilities. Regardless of where we happen to find ourselves in relation to the company, this episode makes clear that pressure can successfully be applied. And it's on all of us to use that collective power for good. As always, if you're finding value in the show, I'd appreciate it if you take a moment to share it with friends, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we welcome your feedback via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. Finally, for now, a quick reminder, I'll be speaking at Imagine AI Live, May 28th through 30th in Las Vegas, the Adapta Summit, August 12th and 13th in São Paulo, Brazil, and the Enterprise Tech Leadership Summit, September 23rd through 25th, again, in Las Vegas. If you're planning to attend any of these events, let's meet up in person. For now, I hope you enjoy this conversation on OpenAI's past, present, and ever-evolving future with ex-OpenAI research scientist, Steven Adler. Steven Adler, former research scientist at OpenAI and now one of the 12 Amici on the recent amicus brief to the Elon versus OpenAI lawsuit. Welcome to The Cognitive Revolution.