Feeling the AGI with Flo Crivello

**Nathan Labenz** (0:00)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello, and welcome back to The Cognitive Revolution. Today, at the end of another dizzying week in AI, and with my voice well on its way to failure, I'm speaking with returning guest Flo Crivello, founder and CEO of Lindy AI, to chew on the week's top story, which was, of course, the situational awareness manuscript from Leopold Oshinbrenner, formerly of the OpenAI Super Alignment Team, and to try to make sense of where we are in the big picture.
If you haven't already read the article or heard the accompanying to our Keshe podcast, the short summary is that AGI is near, super intelligence is likely not far behind, and particularly in the current geopolitical context, it seems almost inevitable that the US and Chinese governments will begin to compete for AI supremacy, likely ending up in an extremely dangerous AI arms race.
Leopold's hope is that the US can maintain enough of a capability lead to give researchers the time they need to solve critical AI safety problems before deploying highly powerful systems. And then, having achieved a decisive strategic advantage, offer China a benefit-sharing deal that they can't afford to refuse.
While such a scenario might have sounded fantastic not long ago, in today's world, try as I might, I can't really find a major hole in Leopold's argument. All the Frontier Lab leaders are talking about AGI on a two-to-five-year timeline. And the US government is already starting to feel the AGI itself. So as a most likely scenario, I think it's a pretty reasonable sketch. And yet, I find the idea of an AI cold war so scary and honestly so tragic that I feel compelled to at least try to imagine an alternative. While it seems almost impossible that the US and China could trust one another enough to work together on such an important issue as AGI development and safety, and while as I've said many times, I would not want to live in Xi's China, I personally would rather take our chances trying to work with the CCP than hoping we can solve AI safety in a super short timeframe under the intense pressure of potentially winner-take-all international competition.
After all, as we talked about last time Flo was on the show, the Chinese government does seem to understand that AI poses a threat, if only to its future political control, and just this week, a Chinese firm has released a Sora-like video generation model, which suggests that Chinese AI research isn't necessarily all that far behind.
All that said, as you'll hear in this conversation, I still have a lot of uncertainty on a number of core issues. Will GPT-5 pose major risks? My best guess is no. GPT-5 class models will probably still be in the sweet spot, where they're powerful enough to be super useful, but still safe enough to deploy, though perhaps not safe enough to open source. Will regulation, like the recently SB 1047, do more harm than good? Flo and I are both longtime libertarians and well aware of the potential, even the likelihood, of unintended and unexpected harms from regulation. But given Frontier Labs' apparent inability to self-regulate, we are both inclined to support it as a starting point. And I, for one, could imagine myself beginning to advocate for an outright pause in further scaling in the not-too-distant future as well. In the meantime, is my adoption-accelerationist, hyperscaling-poser position even coherent? I do still think so, but the argument that adoption inevitably hits the limits of current models, which in turn invites further scaling, can't be dismissed out of hand either.
It is a strange time all around. Flo and I both continue to love exploring AI technology, building products with it, and imagining the many ways it's likely to improve the human condition in the years ahead. And yet, I can't shake the sense that the situation as a whole is at risk of spinning out of control. And not only is there very little that I can do about it, but in some ways I am even contributing to the dynamic.
I certainly don't have all the answers, but I hope that by speaking candidly about these issues, I can play at least some small part in steering the world toward the best possible outcomes. As always, we'd love to hear your thoughts. Do you find value in this sort of real-time reaction episode, even though it raises more questions than it answers? Or would you rather I stay focused on concrete research and applications? I look forward to hearing from you, either via our website, cognitiverevolution.ai, or as always, you can DM me on your favorite social network. For now, I hope you enjoy this weekending conversation with Flo Crivello. Hello, Crivello, CEO of Lindy and feeler of AGI. Welcome back to the Cognitive Revolution.