A Stark Warning From a Former AI Doomer

Cal Newport (0:00)

An argument that I've been making for a long time now is that if we really want to understand what's happening in AI, we have to understand the futurist ideologies that are heavily influencing the top frontier labs, Anthropic and Open AI. The problem is that these ideologies are confusing, and there's rumors swirling around them, and the excerpts we hear can sometimes be absurd, and we think that can't possibly be true. So I thought what would be useful is to bring in an insider to explain to us what is really happening in these labs and what do they truly believe. This is why I'm really happy to have on the show this week, Emile Torres. Emile is a researcher and academic who focuses on eschatological and utopian movements, but what makes Emile really useful this discussion is that they used to be involved in these ideologies. Emile comes from the world of futurist ideologies that believed that superintelligence and AI would either kill us all or redeem humanity with a post-human digital god. But then Emile's moral compass led them out of that world and now Emile has become, in my opinion, one of the most astute chroniclers, expositors and critics of these ideologies. So Emile is going to join me here in a second.

They're going to walk us through what exactly are these different beliefs, how do they intersect, what is their role in the major frontier labs, and why should we be really worried about them. If you have any ambition to understand what's really happening in the world of AI, you need to listen to this conversation with Emile. It is going to, quite frankly, probably shock you, but this is a good shock. This message is one that we all need to hear. So let's get right into it.

Emile, thank you for joining me on the show.

Emile Torres (2:03)

Thanks for inviting me. Great to be here.

Cal Newport (2:05)

I've been looking forward to this conversation for a long time. I think both your story and your research is being too overlooked in our current moment. I think it is very relevant to a lot of issues that people care about or are worried about or don't understand about the current AI situation. I'm excited to really get into both the meat of your work as well as the origin story, which I think is very important. But I thought we should start briefly with definitions just so that everyone is on the same page when we're talking about these various movements. Can you explain what the acronym TESCREL means?

Emile Torres (2:43)

Yes, I can. This was coined by myself during a collaboration with the computer scientist, Timmy Gebru, which we initiated in, I think, late 2022 or so. We were specifically interested in the underlying ideological drivers behind the current race to build ASI or artificial superintelligence.

So TESCREL is a clunky kind of unsightly term, but it does a lot of important explanatory work. So one way to think about the acronym is it's basically just a name for the dominant orthodox worldview within Silicon Valley. That simultaneously highlights a certain intellectual history that is relevant to understanding that worldview. So it explains what the worldview is, where it came from, what shaped it, how it evolved, and so on. And so the acronym itself stands, it represents seven different ideologies.

And the letters appear in roughly the same order as the ideologies emerge chronologically over the past almost 40 years. So the T stands for transhumanism, E for extropianism, then singularitarianism, cosmism, rationalism, affective altruism, and long-termism. A bunch of really big clunky polysyllabic terms. But these ideologies are really fringe within American society. And I think if you were to go and talk to the average person on the street, they would find them to be so bizarre that they'd consider the grand cosmic vision behind these ideologies as basically a non-starter. Nonetheless, they are massively influential within Silicon Valley. Basically, everybody from Elon Musk to Mark Zuckerberg, to Sam Altman, to Dario Amadei are transhumanists. And many people in Silicon Valley don't even call themselves transhumanists. But that's for the same reason that I've never publicly stated I'm around Earth, right? It's just like, what else would I be? Obviously, I'm that. So transhumanism really is the water that people in Silicon Valley swim in, the air that they breathe, and it's also the backbone of the whole test squirrel world view. And so once you understand transhumanism, then all of the other ideologies, some of which are just variants of transhumanism, are sort of easy to grasp.

Cal Newport (5:02)

Well, I want to ask you a history question here, because some of this is new to me. The history of this that I best understand starts with the E. So the history I've been writing or talking about starts with extropiaism becoming a pretty big utopian futurist movement, I guess like the 80s and 90s. From there you get rationalism, sort of simultaneously effective altruism, which starts with a bunch of different goals, embraces ideas from rationalism and long-termism, becomes more focused on a sort of, more of a long-termism, transhuman utopian, anti-AI type of orientation.