Understanding AI "Understanding" with Robert Wright of Nonzero Newsletter & Podcast

**Nathan Labenz** (0:00)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to share a special crossover episode with Robert Wright, publisher of The Nonzero Newsletter and host of The Nonzero Podcast. inspired in part by my recent episode with Martin Casado from A16Z, in which we debated how powerful AI systems are likely to become over the next few years, in this episode, Bob interviews me about the pressing questions surrounding AI development capabilities and risks. We try to be as accessible as possible to a general audience without shying away from technical concepts or critical questions that don't have easy answers. We began with a discussion of what we know about the nature of understanding in large language models and to what extent we can productively compare them to human cognition, and then continued on to discuss a wide range of topics, including the development of multimodal AI systems and their potential for more robust world modeling, the current state and future potential of AI reasoning capabilities, the potential for AI to accelerate scientific discovery and technological progress, the fascinating work being done on AI interpretability, including Anthropix's recent experiments with Golden Gate Clawed, the remarkable degree to which LLMs do understand human values and ethics and why that's definitely not something we should take for granted, why I'm not advocating for a pause in AI development right now, as well as what future developments could cause me to reverse course and begin to do so, the emergence of new AI architectures Let's start with the most important ones, like states-based models and their possible implications. And finally, the pros and cons of open-sourcing powerful AI models, especially as it relates to the increasingly fraught U.S.-China relationship and the risk of an international AI arms race. On this last topic in particular, it's worth noting that just keeping up with AI research and development is a full-time job, and I am by no means an expert in U.S.-China relations. So I'm genuinely very uncertain about what US policy should be toward China with respect to AI. I'm instinctively very skeptical of the idea that we are the good guys, and thus we should be the ones that humanity trusts to develop transformative AI systems first, really for any possible meaning of we, including any of today's leading AI companies or even the West in general. I do believe it's super important to continue to question the wisdom of escalating tensions between major world powers, especially given the potentially transformative nature of near-term AI developments and their obvious military applications. But I do hear compelling arguments in many different directions, and there doesn't seem to be any truly safe path forward. All I know for sure is that this topic is super important, and for that reason, while I normally try really hard to ground my analysis in concrete facts and avoid saying anything that could later be proven wrong, in this particular area I feel it gets worth sharing new ideas and testing new arguments even while I fully expect that my position will continue to evolve. As always, if you're finding value in the show, we'd appreciate it if you'd share it with friends, post online, or leave us a review on Apple or Spotify. And I love hearing from listeners, so please feel free to DM me on your favorite social network anytime. For now, I hope you enjoyed this more speculative than usual crossover episode with Robert Wright of The Nonzero Newsletter and Podcast.

**Robert Wright** (3:39)
Hi, Nathan.

**Nathan Labenz** (3:40)
Hi, Bob.

**Robert Wright** (3:41)
How are you doing?

**Nathan Labenz** (3:43)
I'm doing great. Good morning.

**Robert Wright** (3:45)
Good morning to you. Let me introduce this. I'm Robert Wright, publisher of The Nonzero Newsletter, to which everyone should subscribe. And this is The Nonzero Podcast. You're Nathan Labenz.
You're both an entrepreneur and a podcaster. You founded a company called Waymark, is that right? That uses AI now, didn't originally, uses AI to help companies come up with like video marketing plans and so on. And so you're riding the wave. You're riding the AI wave. And your podcast has really been valuable to me, Cognitive Revolution. Although my comprehension rate is like maybe 65% on a good day, because your podcast is, you know, it's, I don't want to say quite that it's for insiders, but, you know, it's, you tell me what your kind of business model is, but it's pretty high level discussion. You tend to have entrepreneurs and a lot of actual kind of AI engineer type scientists, and you understand all this stuff. And so you speak at a reasonably high level. And the way this conversation came about is I was listening to one of your conversations was fascinated by the subject, because it's something of great interest to me. And I was thinking both, you know, I have some questions I'd like to ask to clarify things. Also, I thought it would be a service if I asked these like dumb questions and kind of brought the conversation down to a level, you know, where mortals can understand it. So that sound like a, did I did I flatter you too much or the opposite?