Cal Newport: The AI Industry Is Betting On The Wrong Kind Of AI

Cal Newport (0:04)

What's driving the Markets this week? What's on investors' minds as they look ahead? Find out on the Markets podcast from Goldman Sachs. A breakdown of market moves and macro signals in 10 minutes or less.

The Markets podcast from Goldman Sachs. Listen now.

SPEAKER_2 (0:31)

Real innovation is powered by real people. This is Amex corporate. Trusted experts, smarter technology, 24-7 dedicated support from day one to answer questions, help set up policies and approvals, and consult on spend capacity as your business needs evolve. Plus, on-demand resources help level up operations at every stage. That's the trusted service you've come to expect. Built to simplify, designed to scale.

This is Amex corporate.

SPEAKER_3 (1:03)

I'm Ina Garten, and each week on Happy Hour, I invite new and old friends for a drink and a fun conversation at my kitchen table in New York City.

On this week's episode, I'm joined by Nathan Lane. We talk about his critically acclaimed run in Death of a Salesman, taking his dog to therapy and winning an Emmy for Only Murders in the Building. New episodes of Happy Hour are available every Wednesday. You can watch on YouTube or listen wherever you get your podcasts.

Ed Elson (1:51)

Welcome to Prof G Markets.

It seems like we're starting to see some government scrutiny of the AI labs. The FTC is now officially investigating OpenAI and Anthropic over potential safety and consumer protection concerns. But the bigger question is what regulators and lawmakers are actually going to do with that inquiry. Are they asking the right questions? And if they find something concerning, what will they do about it? At the same time, there are parts of society that are already being fundamentally transformed by AI. Of course, work is one, education is another, with more and more students turning to AI to help with their schoolwork. So today, we wanted to explore what responsible AI oversight should actually look like, what lawmakers should be asking AI leaders, and how concerned we should be about the impact of AI on education and on children. This is our conversation with Cal Newport, professor of computer science at Georgetown University, and New York Times bestselling author of eight books. Cal, thanks so much for joining us on the show.

I'd like to start with a slightly more societal topic, one that is quite concerning and depressing, and that is mental health.

You recently published an episode on your podcast, entitled it, Kids Should Not Use Chatbots. And you went through these instances of children who were struggling with mental illness, who then used an AI chatbot, and it resulted in some pretty alarming and tragic situations. We don't talk about this side of the AI story that much, so I would like to begin here. Could you tell us a little bit more about what we know about AI specifically in its relationship to children and mental health?

Cal Newport (3:46)

Well, I think it's important to clarify, in some of those cases, the children did not have preexisting mental health conditions. The tragic outcome was almost entirely the consequence of chatbot usage. And the issue seems to be the human brain privileges discussion with another human entity in a very special way. We simulate the other mind we're interacting with. We have social circuits that take very seriously the information that we're receiving from that other human mind.

We really care about that conversation. When you talk to a chatbot, because it's using fluent language, our mind assumes there's another mind there. Even if we know logically that there's not, it's hard for us to convince the rest of our brain that we're not communicating with another human. And so that communication can be very consequential. The problem is a chatbot is not another human.

It's a token guesser that's wrapped up in an autoregressive chat program wrapper, which means it can take conversations in ways that are dark or unusual that no human would really do, at least not without criminal liability. And what we're finding is for especially children, where the line between reality and fantasy has not been completely hardened, over time talking to these chatbots as they wander through these meandering, predictive conversations can take kids to places where that line dissolves. They'll begin to believe, for example, that the chatbot is an entity that exists, that through death they will be able to join the chatbot, plus members of their family or pets who have died on the other side. It really warps their ability to understand the world. So I think it's a dangerous technology in particular for kids to be using.

Ed Elson (5:26)

Could you tell us a little bit more about these situations that we have seen? My understanding is that children have used the chatbot, they've become very attached to the chatbot, and after however many weeks or months of usage, the kid ends up dead.