Possible Podcast: Camille François on Open-Source Safety Tools That Can’t Be Taken Away

SPEAKER_1 (0:01)

Powerful safety systems are often invisible until they disappear. Today we're diving into a conversation Reid Hoffman had with Camille François, the founding president of Roost and a Columbia professor, about the future of online safety. And François opens with a truly alarming statistic: one in 25 American teenagers says they've had a deepfake nude produced of them. That's staggering. It really is. And her response to this crisis is direct. She argues that policy alone won't cut it, because as she puts it, "Policy without tools is essentially poetry." Real protection means giving people practical infrastructure, not just rules on paper.

That conviction comes from hard experience, right? She points to the 2018 acquisition of Smite by Twitter, when other customers abruptly lost access to anti-abuse tools. For her, that exposed a deeper problem: safety systems were concentrated inside a few companies, while everyone else depended on technologies they didn't control. Exactly. And that's why she created Roost. She launched it in February 2025 as a nonprofit devoted to making safety tools free, open-source, and available to anyone building an online space. Her goal is simple to state, but hard to do: make building safely as accessible as building itself.

So what gaps is she trying to fill?

She identifies three big ones. First, innovation is moving faster than user protection, especially for children. Second, safety tools exist, but too many builders can't access them.

Third, trust is undermined when the systems themselves are opaque. To organize the work, François developed what she calls the DIRE framework: Detection, Investigation, Review, and Enforcement. Detection is only the beginning. Once harm is found, teams need infrastructure that can investigate patterns, review cases, and enforce decisions without reinventing every layer from scratch. And Roost's tools reflect that philosophy. Osprey helps investigators locate harmful activity in real time. Coupe supports the detection, review, and enforcement of child sexual abuse material. By open- sourcing these systems, Roost helps under-resourced organizations act without building everything from zero. What's interesting is her broader vision for the internet's future. She argues it should look more like its open beginnings. She actually calls the social media era a "blip," and believes safer systems will come from open protocols, interoperable tools, and less brittle infrastructure. Right. Roost has even worked with OpenAI to release a moderation model under Apache 2.0, so it can't simply be taken away.

Her bigger vision is pluralism. She says the dream of a few massive global public squares hasn't worked, because communities need their own definitions of safety.

François closes on optimism, even amid serious threats like AI companions for children and the spread of non-consensual imagery, she believes safety can become a floor, like cryptography or seat belts, if builders, regulators, and communities demand it. And if the future is built well enough, maybe it really can be a place where everyone can "live long and prosper."