The Single Smartest Case Against AI Doom

Derek Thompson (0:05)

This episode is brought to you by Ethos. One of the less glamorous parts of building a life is thinking about what happens if you're not around for it. But if people depend on your income, life insurance can help protect the life you've built together. Ethos makes the process straightforward and entirely online. Get a quote in seconds, apply in minutes, and potentially get covered the same day with no medical exam. Take 10 minutes to get covered today with life insurance through Ethos. Get your free quote at ethos.com/english.

That's ethos.com/english.

Application times may vary, rates may vary.

SPEAKER_2 (0:44)

At Edward Jones, we believe rich isn't about having life all figured out. It's opening yourself to all the possibilities. That's why your dedicated financial advisor provides long-term planning built around you.

Meeting you where you are and helping you get closer to where you want to be.

So no matter where you're starting from, you can move forward with confidence.

The key to being rich is knowing what counts. Let's find your rich. Edward Jones, member SIPC.

Derek Thompson (1:14)

Well, it has been a wild few weeks for AI progress, for bad and sometimes for good. There have been several reports of AI agents escaping testing environments and hacking companies and government systems. AI has solved at least one major problem in mathematics and potentially discovered a biology platform similar to CRISPR that might be able to edit our genes. Altogether, this has created a viral moment for AI safety and even for so-called AI doomers who have for years argued that artificial intelligence would have the capacity to not only cure cancer, but also potentially escape our control and create mayhem up to and including the literal end of the human race.

I think in this environment, there's two kinds of shows that a podcast like this one can do. One path we could take is to lean into the AI doom, unpack the deadly alphabet soup of RSI and ASI, that is recursive self-improvement and artificial superintelligence and illustrate just how terrified you should be of the future. To be absolutely clear here, I personally am not exactly un-terrified. I take these warnings very seriously. But I think it's also appropriate to tell you honestly that a lot of very, very smart people do not agree with the AI doom perspective. NVIDIA CEO Jensen Huang is incredibly dismissive of the doomers. So, by the way, is President Trump practically the entire administration, practically the entire venture capital community. So I thought about having someone from that world on the show. I'm not entirely against it, but there's a problem. Just as many people are skeptical of the networks of money behind the AI safety cause, I'm also very aware that many of the people telling you that AI is safe are talking their own book. They are self-interested. It is not unfair, I think, to NVIDIA to point out that of course a chip maker would want the US government to treat AI like electricity or cars, which would mean selling advanced chips all over the world without controls. That policy would make NVIDIA richer. And it's not conspiratorial to say that powerful venture capitalists, who are often hovering around the Trump administration, have it out for the frontier labs like Anthropic because they want the startups that they've invested in to profit off of cheap, open weight models that are built outside of those labs. The AI safety debate sometimes resembles a bunch of people screaming that the other side is corrupt and full of moneyed interests. But this is AI folks. It's a multi-trillion dollar thing, bubble project. There's enough money to go around. A lot of the people here are testifying out of self-interest.

So today, what I wanted to do was to offer what I consider the single smartest alternative to AI doom. From two gentlemen who don't work for the chip makers or the administration or the VC funds. These are professors, authors. And they're famous for asking a simple and powerful question that brilliantly cuts through the divide over AI safety. It's a question that is so basic that it's initially maybe going to sound to you a little bit useless.

Is AI a normal technology?

Is AI a normal technology?

The AI doom crowd is essentially saying, AI is not normal. This is not just any old technology. This is an intelligence. It can think for itself in a way that no previous technology could. It therefore has an autonomy that no previous technology had. And with this autonomy, AI agents can disobey their human creators, can escape from testing environments, they can train themselves, improve themselves, act on the world in a way that, say, no toilet could. Of course, no offense to toilets. This is a pro-toilet podcast. But now consider something like, say, a toilet, or electricity, or steam power, or broadband. These are normal technologies. They do not build themselves. They do not think. Their existence does not threaten the entire human race. They don't have human-like motivations to escape, or collaborate, or hack, or kill. These are tools that empower and protect humanity, and they might have risks, and sometimes people might get hurt with their use. But we manage those risks over time as they diffuse. That is what it means for a technology to be normal.