The Risks of Too Much AI: Fortune’s Jeremy Kahn

**Sam Ransbotham** (0:03)
What will the AI revolution mean for jobs, for our kids, for our relationships, and our daily lives? A new podcast, Pioneers of AI, is your trusted guide to this emerging technology. Host Rana Al-Kaoubi is an AI scientist, entrepreneur, author, and investor, exploring all the opportunities and questions AI brings into our lives. Listen to Pioneers of AI with new episodes every Wednesday, wherever you tune in.
Hey, listeners. When it comes to AI, it's hard to decipher what's real, what's hype, and what you need to know. To help make sense of it all, we recommend you listen to the TED AI Show. Host, creative technologist and AI innovator, Bilal Vosadou, sits down with guests for a deep dive into the thrilling world of tech and its effect on business, education, and more. Each episode will leave you more informed with tools to help you navigate the future of tech. Get ahead of the curve and check out the TED AI Show wherever you get your podcast.

**Shervin Khodabandeh** (1:19)
Stay tuned after today's episode to hear Sam and I break down the key points made by our guest.

**Sam Ransbotham** (1:27)
In what ways are we over reliant on AI technologies? On today's episode, we chat with a journalist who shares some of his fears about AI.

**Jeremy Kahn** (1:37)
I'm Jeremy Kahn from Fortune, and you're listening to Me, Myself, and AI.

**Sam Ransbotham** (1:43)
Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, Professor of Analytics at Boston College. I'm also the AI and Business Strategy guest editor at MIT Sloan Management Review.

**Shervin Khodabandeh** (2:02)
I'm Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

**Sam Ransbotham** (2:29)
Hey, everyone. Thanks for joining us today. Shervin and I are excited to be talking with Jeremy Kahn. He's the AI editor at Fortune Magazine. At Fortune, he writes his weekly newsletter, Eye on AI. He's also the author of a book, Mastering AI, A Survival Guide to Our Superpowered Future. Jeremy, we're looking forward to talking today.

**Jeremy Kahn** (2:49)
It's great to be here.

**Sam Ransbotham** (2:51)
So I'll raise a complaint first. I had planned to just skim the book and just try to pull out a highlight or two, but it was interesting and so it unfortunately cost me a lot more hours this weekend preparing than I planned. So that's my angry start to this podcast episode.

**Shervin Khodabandeh** (3:06)
That's a good complaint. This is what Sam's compliments always sound like.

**Jeremy Kahn** (3:10)
I was going to say sorry, not sorry on that one.

**Sam Ransbotham** (3:13)
Well, but maybe go negative here. We heard a ton about the benefits of AI, but let's start with some of these risks that you foresee. I thought you did a nice job outlining those. For example, you caution against letting AI cognitive prowess diminish our own prowess. Can you walk us through how some powerful tool might diminish our own human abilities?

**Jeremy Kahn** (3:33)
Yeah, absolutely. It's one of the things I'm concerned about is that this technology is in some way so easy to use and so seductive that we'll sort of overuse it and as a result lose some of our important cognitive abilities. What I worry about is just critical thinking. I mean, anyone who's used one of these chatbots from OpenAI or Anthropic or any of the others, it gives you a very fluid, plausible, pat answer and a very capsule summary of whatever information you want. And I think it's just really easy to kind of take that answer and not think too hard about it. And go off and say, oh, I've got the answer. And in the way these systems are currently designed, there's no real prompting of the user to think about the provenance of the information that they're receiving. And in fact, in some cases, the way these things are designed, there's no way the system itself doesn't really understand the provenance of the information it's giving you. So it can't actually tell you how it knows what it knows.

**Sam Ransbotham** (4:28)
Explain provenance, though, to start with.

**Jeremy Kahn** (4:30)
Yeah. So provenance, I mean, what was the source of this information? Any particular fact it's telling you, where did that come from or its analysis, how did it arrive at that? People said some of these things about Google. When Google came along, there were some criticism that Google was going to make us stupid. But one of the good things about Google is at least you have these links. So you have to think about where's the information coming from. At least there's something there in the UX that prompts you to think about the source of information. If you want to know more than just a few lines that the link displays, you have to click the link. You don't have to do any of that with a generative AI based chatbot response. And I worry about that. I worry we're just not going to think too hard about where's this information come from. So that's one of the ones I'm concerned about. The other is the tendency with these LLMs to think that writing and thinking are completely separate activities. I'll just give the system a few bullet points and it will write the whole essay for me. And it will do all the hard work of thinking through the arguments. And I worry about that because again, I think writing and thinking are not actually separable activities. That it's really through writing that we refine a lot of our reasoning. Very often you find holes in your own argument when you actually go down and write something long form that you haven't thought of when you just outlined a few bullet points. And again, there's some tendencies that were already there with other technologies. PowerPoint most famously had this tendency to reduce things to bullet points, often with some negative repercussions. That's why Jeff Bezos had famously kind of banned PowerPoint presentations at Amazon because he was very worried about the kind of sloppy thinking that came in when you just had people delineate their thoughts and bullet points.