The AI Whistleblower Initiative: Supporting AGI Insiders When It Matters Most, w/ founder Karl Koch

**Nathan Labenz** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, my guest is Karl Koch, founder and managing director of the AI Whistleblower Initiative, a nonprofit dedicated to supporting concerned insiders at the frontier of AI development. I am particularly passionate about this work because I would have loved to have had this kind of support a couple of years ago, when, as longtime listeners will know, I tried to raise concerns about the size and quality of the GPT-4 Red Team project and the ineffectiveness of the nascent safety measures that OpenAI had developed at the time. Then, no such support existed, and so I consulted friends in the AI safety community before ultimately deciding to escalate my concerns to OpenAI's board, for which I was subsequently dismissed from the program. That experience left me acutely aware of how difficult it is for insiders to navigate these situations, and has motivated me to support this project with a mix of modest personal donations and behind-the-scenes fundraising help and a bit of ad-hoc volunteer work over the last six months. Along the way, I have been consistently impressed by the seriousness of Karl's thinking and the maturity of his approach. Aspifit is an organization that aims to help people in truly critical moments in their careers, when the stakes have never been higher for them personally and potentially for society as a whole. They are taking care to lay a foundation of understanding and infrastructure now so that insiders can trust them if and when that pivotal moment comes. The first critical investment they've made is an extensive research and understanding. By talking to over 100 governance researchers and surveying employees at Frontier AI Developers, they've developed a deep understanding of the barriers that potential whistleblowers face. The majority of Frontier Lab insiders, as it turns out, don't even know if their companies have internal whistleblowing policies, let alone understand what protections they offer. The legal landscape, unfortunately, doesn't help much either. The EU will begin to protect AI whistleblowers starting in 2026, but US law remains a patchwork, with proposed legislation, like the AI Whistleblower Protection Act, still pending. Strikingly, roughly half of survey respondents expressed a lack of confidence in their own ability to determine whether specific observations constitute a serious cause for concern. And literally 100% lacked confidence that regulators would understand, let alone be able to act effectively on their concerns. Meanwhile, over 90% couldn't name a single whistleblower support organization. All this, plus well-known cases where people like Leopold Aschenbrenner were fired for breaking chain of command and going directly to the OpenAI Board with security concerns, creates a highly uncertain and risky context for such high-stakes decisions, in which people with serious safety concerns are left to think through the nuanced costs and benefits of internal escalation versus going to regulators versus leaking to the press almost entirely on their own. It is an extremely stressful position to be in and not conducive to the best possible decision-making.
The good news is that the AI Whistleblower Initiative offers several forms of support. Their third opinion service, which you can find online at aiwi.org, allows insiders to anonymously reach out via a Tor-based open source tool which, Karl notes, is pen-tested with security reports published openly for scrutiny and verification to get help identifying and anonymously contacting independent experts who can answer questions without requiring you to share confidential information or even reveal where you work. For those concerned about digital privacy, they provide a digital privacy guide and, in select cases, hardened devices with specific operating system setups for highly secure communication. And, if insiders need concerns justified, they also connect people with specialized and experienced whistleblowing support organizations, including the Signals Network, psst.org, and the Government Accountability Project, which can provide pro bono legal counsel, psychological counseling, and guidance throughout the process, all without pressure to disclose any information. Crucially, in some cases, they can even help arrange financing to cover legal costs, which can easily add up in cases that end up in any form of litigation.
That a nonprofit stands ready to invest this seriously in any concerned insider that needs help may strike some people as excessive today. But considering that we're talking about perhaps just hundreds or maybe low thousands of people globally who are positioned to spot and raise critical concerns over the next few years, of which I'd bet only a few dozen will ever find themselves in a position to seriously consider sounding alarms, I think this sort of care and support is absolutely worthwhile. Most recently, Karl and team have launched the Publish Your Policies campaign, online at publishyourpolicies.org, calling on frontier AI companies to make their internal whistleblowing policies public. This is actually standard practice in many industries, but interestingly in the AI space, only OpenAI has done any version of this, with their raising concerns policy, which they published only after Daniel Cocotello and others revealed OpenAI's use of extensive nondisparagement agreements to keep former employees from publicly criticizing the company. Daniel, by the way, is joined by other former AI insiders, luminaries like Stuart Russell and Lawrence Lessig, and yours truly in signing on to the Publish Your Policies campaign. Of course, publishing corporate policies doesn't obviate the need for proper legal protections, which Karl strongly advocates for as well. But at a minimum, it would help insiders understand their rights and options, enable public scrutiny, and ultimately create accountability that benefits everyone. If you work at a Frontier AI company, Karl encourages you to ask your management to consider publishing their whistleblower policies. If enough people ask these questions now, I wouldn't be surprised if it becomes another dimension of the intense competition between Frontier AI developers for top research talent. And that could ultimately mean that companies even begin to collect and publish data on things like how many reports they receive, their response timelines, retaliation complaints, appeal rates, and whistleblower satisfaction scores. All of which would benefit everyone. Regardless of what company leadership decides to do, Karl's message for insiders is this. Support is available at every stage. Whether you're considering internal escalation, thinking about approaching regulators, or even contemplating public disclosure, you can reach out completely anonymously without sharing any confidential information just to understand your options. And the AI Whistleblower Initiative can help you get expert perspective on what you're seeing, connect you with legal counsel and experienced guidance, and perhaps even help finance your case. So don't wait until you're deep into a crisis. And know that you don't have to face this alone. With that, I hope you enjoy, and I encourage you to share with friends who work at Frontier AI Developers, this in-depth conversation about the challenges that concerned AI insiders face, and the support that's available to make sure that those who are building our AI future can safely speak up when it matters most. With Karl Koch of the AI Whistleblower Initiative. Karl Koch, Managing Director at the AI Whistleblower Initiative. Welcome to the Cognitive Revolution.