Stopping The AI Safety Cult ft. Adam Becker & Cal Newport

Ed Zitron (0:01)

Causa Media My name is Ed Zitron, and this is Better Offline.

I am back and I am announcing that we will be speeding up the process of researching powerful and dangerous podcasting technology. My researchers are spending over $200 million a day on GPU compute trying to create and generate a recursive self-improving Zitron and no amount of pressure from world governments will stop me. But before my greatest work is complete, I must continue to record. And today we've got an absolute barn burner. I'm joined by the claviculars of computer science, by which I mean Professor Cal Newport, an astrophysicist and author of More, Everything Forever, Adam Becker. Fellas, welcome back to the show.

Adam Becker (0:50)

Thanks for having us.

Cal Newport (0:52)

Yeah, I'm terrified of the idea of recursively self-improved. That means your newsletters will be 200,000 words long now? Is that what we'll expect?

Ed Zitron (1:01)

Yeah, they'll either be 200,000 words or 100 words long.

Cal Newport (1:05)

Oh, maybe it's not the worst thing then. All right, keep going.

Ed Zitron (1:07)

When the perfect matrix of Zitron is complete. Well, today though, this was inspired by your wonderful piece, Cal in New York Times, about the rationalists. And we are going to start very simple. Who are these people and why are they suddenly notable?

Cal Newport (1:23)

Well, I'm probably going to actually throw the baton to Adam here in a second to give the history of this group because he and his book was one of my primary sources in trying to understand them. But let me just set the stage for why I was writing about them.

I don't think people realize just how weird some of the lead figures are, who are at, in particular, OpenAI, Anthropic and DeepMind. And I don't think people realized that until last week, when you had a whole group of 20-somethings, casually debating in their lunch hour, whether it's 10% or 50% chance that we're all going to die within the next couple of years. And that, I think, caught a lot of people's attention. I'm like, wait a second. What is going on over there in those labs? Why is this type of rhetoric only coming out of those particular labs? Why is Jensen Wong think that this is ridiculous? Why is China laughing at this narrative? There's something going on here that's not right. I think people are starting to wake up to this. And I think it really is this family of futurist philosophies that come out of the Bay Area that have mutated and split in various ways, that are really defining our current moment, and no one really realizes it. But Adam spent years, God help him, immersed in this world, has read everything they've written. He knows the history well. So actually, Adam, I'll throw the baton to you. Maybe you can give us a story of rationalism and effective altruism and where they came from and how they've split. To me, it's a fascinating, odd story.

Adam Becker (2:51)

Oh, God. I mean, my standard response when people ask me this is, you know, well, I could write a whole book about it and I did. And you know, I'm not going to get into the same level of detail I do in the book. But like, basically, this is something that was spawned out of transhumanist message boards and email lists in the 1990s.

Ed Zitron (3:13)

And what is transhumanist?

Adam Becker (3:15)

Yeah, that's a good question. Transhumanists, I guess the shortest good definition I could give you is transhumanists are people who don't want to be human anymore. They want to use machines and especially computers to basically escape human bodies and solve the problem of death.

Ed Zitron (3:41)

Okay.

Cal Newport (3:42)

Yeah, so those guys.

Adam Becker (3:44)

Yeah, normal shit. Normal shit, normal dudes.

Cal Newport (3:46)

The normal way to do things.

Adam Becker (3:48)

Yeah, exactly, yeah.

And so these are people who are really excited about the prospect of AI, the prospect of nanotechnology that could basically serve as a kind of magic to do whatever they wanted. This was the 90s, nanotech was like the hot thing. And it was a bunch of ideas that basically come from science fiction.

Ed Zitron (4:18)

Right.

Adam Becker (4:19)

They read too much sci-fi and confused it with reality and thought, oh, well, because all of the sci-fi I read takes place in the future, and all of the sci-fi I read has super intelligent machines in it, and nanotech and people uploading their brains into computers, that must be what's going to happen in the future if we just work hard and wish hard enough for it.