Confronting the Intelligence Curse, w/ Luke Drago of Workshop Labs, from the FLI Podcast

**Nathan Labenz** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, I'm sharing a special cross-post episode from the Future of Life Institute podcast, hosted by Gus Docker and featuring Luke Drago, co-author of The Intelligence Curse and co-founder of Workshop Labs. I wanted to bring this conversation to your feed because it highlights a critical question that I think society should be grappling with much more than we currently are. Is it wise to design AI systems, to compete directly with and potentially replace humans as economic actors? Personally, I'm relatively optimistic about humanity's ability to adapt to the social and economic changes that will come with AI. And I tend to worry much more about catastrophic scenarios where we lose control of AI systems entirely. But this conversation did force me to confront the possibility that things might still go seriously wrong even if we do manage to solve the alignment problem.
Luke focuses on a particular failure mode which he calls the Intelligence Curse. This concept echoes the Resource Curse phenomenon that we see in some resource-rich, but underdeveloped countries today, where an extractive elite manages to maintain power without democratic legitimacy or even much in the way of cultivating the productivity of its own population simply because they control key resources. By analogy, in a future where AI systems power the economy and human labor is no longer much of a bargaining chip, whoever controls the AI could have a dangerous level of power. I have to say, for as much as I'm hopeful that the AI Revolution can finally free people from doing work they don't enjoy, this dystopian vision is a pretty natural extrapolation from what happens in today's world when human workers are rendered economically uncompetitive for whatever reason. And as we've seen even in many parts of the United States, the results are not pretty nor without consequences for the rest of the country and the world. Listen to this episode and I think you'll have to agree that at a minimum, if AI is going to get anywhere near as powerful as I and all the Frontier Lab leaders seem to think it will, we are going to face a massive challenge. Luke, to his credit, does have some very interesting ideas about what we can and should do to solve this problem. At the societal level, he recommends investments in open-source AI to commoditize the intelligence layer and prevent excessive economic and political rents from flowing to model owners. For companies, he emphasizes the need to design AI systems that empower individual users, while allowing them to retain control over their economically valuable data. And for individuals, he suggests guarding your valuable know-how carefully, developing end-of-one career paths, and chasing moonshot projects sooner rather than later. I'm excited to see how Workshop Labs delivers on this vision, and I hope to do a full episode with them when they launch to the public in the coming months. For now, I hope you enjoy this conversation about the intelligence curse and how we might break it. From the Future of Life Institute Podcast, with Gus Docker and Luke Drago.

**Gus Docker** (3:08)
Welcome to the Future of Life Institute Podcast. My name is Gus Docker, and I'm here with Luke Drago. Luke, welcome to the podcast.

**Luke Drago** (3:15)
It's great to be here. Thanks for having me.

**Gus Docker** (3:17)
Great. So you have this essay series on the intelligence curse. And maybe we should just start at the very core of that and ask, what is the intelligence curse?

**Luke Drago** (3:27)
Yeah. So I'd summarize the intelligence curse pretty simply. The idea is that if you have non-human factors of production and they become your dominant source of production, your incentives aren't to invest in your people. And this sounds very abstract. What does it mean to have a non-human factor of production? And what does it mean that we can build things that actually replace us? And why doesn't this just result in like AGI utopia? But I think we have some concrete examples. And one of the ones that we point to in the essay, and what we actually name the effect after, is the resource curse, where there are states that rely primarily or have a significant amount of their income that come through oil revenues as opposed to investment in their people. And what you end up seeing is because investments in oil produce a greater return than investments in their people, those states oftentimes funnel money towards the oil investment as opposed to their people. The result of this is a worse quality of life for their people who have much less power because at the core, your ability to produce value is a core part of your bargaining chip in society.