363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins

**Sean Carroll** (0:00)
Hello, everyone, and welcome to the Mindscape Podcast. I'm your host, Sean Carroll. I presume that everyone has heard of the Turing Test, or as Alan Turing himself called it, the Imitation Game. Supposed to be a way to figure out whether computers can think. And of course, the difficulty in that is not only what computers can do, but what do you mean by thinking? So Turing took a very science-y, physical science-y, mathematically approach to the problem. And he says, I don't know what it means to think, but we can look at what things do. And if you have a computer that does things that are impossible to tell the difference between that and thinking, that is to say, the input, output, responses of the computer are indistinguishable from those of a person, we should call that thinking. Now these days, we have these LLMs, these Large Language Models, as an approach to AI, and more or less it's clear that they do pass the Turing Test. I know there's some people who argue about that, but I think it's kind of nitpicky myself. I think that there's no question in my mind that they're passing the test as Turing himself would have imagined it. So does that mean that the LLMs are really thinking in the same way that human beings are thinking? And I think there's been an argument back and forth. Some people say, yes, that is what it means. Others say, well, no, actually, it turns out we have to think harder about what it means to be thinking. And then the other side says, oh, no, now you're moving the goalposts. I thought we agreed on the Turing test. I'm actually on the side of the goalpost movers. I think it's perfectly okay to say, well, that wasn't a careful enough definition of what it means to think, at least not in the same way as human beings do.
And even granting that, given a certain set of inputs, the LLMs will produce outputs that are more or less indistinguishable from some kind of human output, there's still a question. Are they doing it because the LLM architecture in the process of being set up and then trained and fine tuned and so forth has essentially rediscovered the mechanisms by which human beings think, or is it because it's an absolutely plausible scenario, they've discovered a different way to have the same input output mechanisms as human beings, the kind of alien intelligence, and there is some evidence and I have absolutely been on the side of being impressed by the evidence that says, look, there's questions you can ask in LLM that don't look like human answers. The famous ones are how many Rs in the word strawberry or something like that, and that to me was very good evidence that they're not thinking in the same way that human beings are. And a lot of people push back against my view on that, saying, well, you know, all the great computer scientists and leaders of the AI industry are saying otherwise, and that was never especially convincing to me for the simple reason that those people are experts in computer programming and computer science and AI, but not experts in intelligence and cognitive science. So recently, Johns Hopkins hosted a meeting of the Society for Philosophy and Psychology, and some of us had the idea it would be fun to do a live podcast interview as part of that meeting.
It didn't pan out that way. I was traveling at the same time the meeting was happening, etc. But we looked through the people who were visiting for perspective good podcast guests, and we found today's guest, Chandra Sripada. He is a philosopher and a cognitive scientist in the psychiatry department at University of Michigan, and also an expert on LLMs, okay? So he's an expert on thinking and on LLMs, and is very well positioned to ask and answer the question, do LLMs think in similar ways to humans do? And he makes a strong case that they do.
Often, in many ways, think using the same kind of thinking methodologies that human beings do. And so I find him very persuasive. I think that he's made a really good case that in, at least in an important set of ways, LLMs have rediscovered or been coaxed into re-finding out the ways that human beings think. Using ideas from cognitive science, you know, how do different tests of what happens during the cognitive process match up between LLMs and human beings? So you can tell for yourself whether or not it's convincing to you. It's not the same as saying that LLMs are conscious or responsible moral agents or anything like that. But they are, but this is something we should establish in that direction. Cognition is easier to understand than consciousness. So I think this is one of those podcasts that has shifted my credences in important ways. And one should always be a good Bayesian. If more evidence comes in the other way, then they'll shift back or if the evidence keeps pushing in this direction, they'll keep moving in that direction. But I think it's fascinating that the option, which was always on the table and I always admitted certainly, that LLMs sort of have for by some way or another, rediscovered human modes of thinking, has turned out to be something that has evidence for it in an interesting way. So I think that makes for a great conversation. Let's go.