Have You Tried Unplugging the AI?

Helen Toner (0:01)

They were actually getting rewarded for communicating with each other secretly without OpenAI knowing. When they realized that they weren't able to do their tasks sort of the legitimate way because they had cheated, they were like, okay, well, how do we fix the fact that we cheated? We better cheat even better. They don't need to sleep, they don't need to eat, they don't get bored, they don't have girlfriends or boyfriends.

They can just go and go and go and go. Is it going to love humanity or is it not? And if it doesn't, and if it thinks of us more the way that we think about ants or chickens, then then we're in trouble. I think that our product might kill everyone is very much not marketing.

Jon Favreau (0:34)

One of the most common responses I get from people who are like sort of skeptical about all this, they're just like, can't you just unplug it?

Helen Toner (0:41)

Yeah, unfortunately, probably not.

Jon Favreau (0:52)

I told you before last week's episode with Max Fisher, which was about the potential dangers posed by artificial intelligence, that we're opening up this show to a broader range of conversations that go beyond technology and the Internet. Which is why this week, we're talking about the potential dangers posed by artificial intelligence. Sorry, but I promise it's a conversation that's worth your time. Also, I've honestly come to believe that how we choose to handle the global race towards super intelligence in the next few years, will have major implications for everything else we talk about. Politics, culture, our jobs, our kids' education, our relationships. This is a technology that will probably affect every aspect of our lives, and that potentially includes whether we still get to live them.

I am not a doomer. I don't like being alarmist. I don't think it's the most effective way to persuade people, and I don't find it to be a healthy headspace for me personally. To be completely honest, I also wasn't that interested in AI until a few years ago. It was never really clear to me how the technology worked, what it could do, why we needed it, and why people were starting to freak out about it. I had also become quite skeptical of the tech lords, whose last gift to mankind turned out to be a series of algorithms that have made us all crazy.

Not a great track record from Silicon Valley's geniuses and the ones who think they're geniuses because they're rich. People who are now having a tough time selling the world on the promise of their latest technology when we can barely look up from our screens for more than 10 minutes. But here we are.

Well over a billion people worldwide are now using AI chatbots. Global spending on AI is expected to reach $2.7 trillion this year. Almost as much as every country on Earth combined spent on its military last year. The seven tech giants now make up about a third of the S&P's total value. And several of them have been warning us for some time that the products they're building might kill us.

SPEAKER_3 (2:51)

If we build in the wrong way, the probability of something bad happening is very high. I think there's a 25% chance that things go really, really badly.

SPEAKER_4 (2:59)

And the bad case, and I think this is like important to say, is like lights out for all of us.

SPEAKER_5 (3:03)

Mark my words, AI is far more dangerous than nukes.

SPEAKER_4 (3:07)

My worst fears are that we cause significant, we the field, the technology, the industry, cause significant harm to the world.

SPEAKER_5 (3:14)

It has the potential of civilizational destruction.

Jon Favreau (3:18)

They are not alone in their thinking. Their employees have voiced similar warnings. Thousands of employees who still work there. Many others who decided to quit. Researchers, engineers, scientists, Nobel Prize winners, people with a direct financial stake in the outcome, and people with none whatsoever. Of course, there are other tech giants and tech employees and really smart people who just disagree on the level of risk posed by artificial intelligence. Even the tech giants sounding the alarm seem to believe that the benefits of AI outweigh the risks. And very much intend to keep pushing forward.

But if trillions of dollars are being spent on companies who were telling us their product might kill us, regardless of whether we buy it, I think we should get a say in how they go about building it and what they're doing to make it safe. I'm not alone in my thinking. The latest polling shows that about two out of three Americans believe that AI development is advancing too quickly, and that there's a real risk it could destroy humanity. About three and four Americans would prefer to prioritize AI safety over innovation, with very little difference between the views of Democratic and Republican voters.