Is an AI Arms Race Inevitable? with Robert Wright of Nonzero Newsletter & Podcast

**Nathan Labenz** (0:01)
Hello, and welcome back to The Cognitive Revolution. Today, I'm pleased to share a special crossover episode with Bob Wright, publisher of the Nonzero Newsletter and podcast. Bob recently wrote an important piece called AI Surrenders to the Military-Industrial Complex, about how leading AI developers, including Anthropic and Meta, are now beginning to partner with the US military. He shared the piece with me just before I attended The Curve, an immersive conference hosted in Berkeley this past weekend, which brought AI obsessives from academia, industry, AI safety nonprofits, and the national security community together for presentations and conversations on a wide range of AI topics, including the future of scaling laws, the latest findings in alignment research, and growing challenges in international relations. There, I had a chance to participate in a fascinating AI war game exercise, organized in part by recent guest, Daniel Cocotello, and designed to explore how critical actors might interact in the context of a fast AI takeoff in 2027 As you'll hear, our simulation included a war between the US and China, despite the fact that the AIs themselves were relatively benign. With that experience in mind and drawing on Bob's expertise in foreign policy, in this conversation, we attempt to characterize the growing hawkishness toward China in AI policy circles. Explore why this stance has become so dominant, particularly among AI safety groups. Assess the wisdom of US strategy in what Bob provocatively, but I think ultimately reasonably calls the chip war. Question whether the militarization of AI development is truly inevitable or wise. Critique influential recent essays by Dario Amadei and Leopold Aschenbrenner. And ultimately look beyond today's prevailing narratives to identify potential paths forward that could minimize risk of US-China conflict. Along the way, Bob and I each make very clear that we are not fans of Xi, would not want to live under the current Chinese government's censorship regime, absolutely disapprove of many of China's policies, both domestically and internationally, do not want to be naive in our approach to China, and candidly do not have great answers. And yet we both feel that our time is well spent searching for a better plan than the one we're currently hearing from both the incoming US president and major AI lab leaders, which amounts to declaring an AI arms race and attempting to build sufficient AI-powered advantage to back China into a corner and ultimately force them to submit to American hegemony. This, without a doubt, is an extremely risky strategy, and one that I fear will leave all of humanity losers. As always, we invite your feedback, particularly if you have ideas for how to build trust between great powers, create novel mechanisms for AI governance, or otherwise nudge history in a positive direction, please let us know either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. I am not leaving Twitter, but I did just join Blue Sky, and my handle there is nathan.labenz.
Finally, before we begin, a quick note on format. Bob is publishing the first half of this episode publicly and placing the second half behind a paywall for his subscribers only. We have obviously taken a different approach with The Cognitive Revolution, relying on commercial sponsorship to make all of our content open and free. We are tremendously grateful to our sponsors, who clearly value a thinking audience and have never attempted to influence our editorial direction. Without them, we simply could not sustain our pace of two episodes per week. That said, we have received some requests recently for an ad-free version, and so I'm wondering how much interest there would be in such an offer. If you'd be interested in a paid ad-free feed, please reach out and let us know. If there is serious interest, we'll look into that possibility. With that, I want to wish our American listeners a happy Thanksgiving, and to challenge all of you to spend a little more time thinking about how you, as an individual, can work toward a more peaceful, positive future. Now, here's my conversation with Bob Wright on the trajectory of AI advances, US-China relations, and how we might avoid sleepwalking into catastrophic conflict.

**Robert Wright** (4:15)
Hi, Nathan.

**Nathan Labenz** (4:16)
Hi, Bob.

**Robert Wright** (4:17)
How are you doing?

**Nathan Labenz** (4:19)
You know, never a dull moment in the AI game, that's for sure.

**Robert Wright** (4:22)
Or the world generally. And actually, we're going to talk about the intersection of the two. So, you know, I'm really looking forward to this. Let me introduce us. I'm Robert Wright, publisher of Nonzero Newsletter. This is the Nonzero Podcast. You're on Nathan Labenz, host of the highly regarded Cognitive Revolution podcast, which is one of my three go-to podcasts on AI. It's been very valuable for me. You're also founder of a company called Waymark that does video ads for small companies and employs AI in that process to make it cost effective for the companies in question. Let me, if you'll indulge me, give like the background story to this conversation. So, a couple weeks ago, I published in my Nonzero Newsletter a story called AI Surrenders to the Military Industrial Complex. The news peg, as we used to say in journalism, was just that a couple of companies, Anthropic and Meta, had kind of crossed a line. They had decided that they were going to do business with the US military, involving their large language models, and also involving Palantir, by the way, a well-known Silicon Valley defense contractor, that had focused more on predictive than generative AI traditionally, but I'm sure it's open to all forms of Pentagon revenue. And, you know, I DM'd you about the piece. I sent you the link, thought you might be interested in it. And I should say, the part of the story that I thought was most important, actually, was not the news, but kind of why some of it surprised people and why they shouldn't have been surprised. And I'm thinking in particular about Anthropic. As you know, Anthropic has a reputation for being particularly safety conscious, concerned about harms done by AI. And of course, the military is in the business of harming people. Now, you know, I don't say that pejoratively. I grew up in a military family. I understand why countries need militaries. I actually have a lot of respect for the military as an institution.