Uncanny AI: Why AI bots remember random, sometimes useless information

**SPEAKER_1** (0:00)
This podcast is supported by Raymond James, a financial firm offering wealth management, banking, and capital market services that are inspired by people.
Before Raymond James' financial advisors build plans, they build relationships so they can craft individual strategies designed to achieve priorities and pursue what's possible.
That's the power of personal. Disclosures at raymondjames.com. Raymond James and Associates Inc., member NYSE, SIPIC.

**SPEAKER_2** (0:29)
This Marketplace podcast is presented by Tomorrow's Cure. If our episodes on the intersection of AI and healthcare left you wanting to learn more about how technology is transforming medicine, Tomorrow's Cure is for you. It's the chart-topping 2025 AmbiAward finalist podcast from Mayo Clinic. Back for a brand new season with new host award-winning journalist Lindsay Sievert, Tomorrow's Cure explores the innovations changing the healthcare landscape. Featuring conversations with leading physicians, researchers and medical experts, the new season examines everything from AI-powered diagnostics and cutting-edge cancer therapies to surgical technologies improving patient care today.
Not sure where to start? Listen to the season premiere featuring MD Anderson radiation physicist Dr. Paige Taylor and Mayo Clinic radiation oncologist Dr. Adam Holtzman. They discuss why carbon ion therapy is generating excitement in the medical community and what it could mean for the future of precision cancer treatment. So go ahead, follow tomorrow's cure on Apple Podcasts, Spotify or wherever you get your podcasts.

**Meghan McCarty Carino** (1:34)
Chatbots remember the darndest things. From American Public Media, this is Marketplace Tech. I'm Meghan McCarty Carino.
I've been using chatbots pretty intensively the last couple years, and there are some details I've shared about myself they seem to think are really important. For instance, Claude is obsessed with the fact that I sometimes get up for work at 4 a.m. It comes up constantly in our chats. Apropos of nothing. So why do large language models latch on to these random tidbits? It's the first topic in our new series, Uncanny AI.
Those moments when it becomes clear, AI doesn't think like us. Here to help us unpack what might be going on is Janelle Shane, a laser scientist who writes the AI Weirdness blog.

**Janelle Shane** (2:38)
These models have a couple of ways of keeping track of information long-term. So there's the chat history itself, and it's using that as an input to whatever it's generating next. But things can get buried in the long, deep history.
So a lot of these models have another way of keeping track of information, which is a separate memory file in which the model or sometimes you or both will insert things that you think are important that you want to persist. And the 4AM is in that memory file. These models are trained to remember to use stuff from the memory file. So that may be part of why it's persisting. Of course, there's the question of, but then why is it bringing up in a weird context?

**Meghan McCarty Carino** (3:27)
Yeah, it sort of reminds me of like, you know, a businessman who read like how to win friends and influence people and like memorized all the, its colleagues, you know, children's names or something and just brings them up randomly. Yeah, for some reason that seems to have become a bullet point in the story of Meghan is the 4 a.m. wake up.

**Janelle Shane** (3:51)
Yes, you might be able to go in and edit it because these are, you know, it doesn't really have the context of knowing what's appropriate or proportional or not. It's, you know, it's almost like this is an element in a story you're telling with it. And in fact, it is trained on a lot of fiction. So some of these can start to act like plot elements, like, oh, don't forget the 4 a.m. This will be important later.

**Meghan McCarty Carino** (4:17)
It's like Chekhov's gun. 4 a.m. is my Chekhov's gun. Many things that I have told Claude are Chekhov's gun. The thing that sort of has to come back later in the plot.

**Janelle Shane** (4:28)
Exactly.

**Meghan McCarty Carino** (4:30)
So, there's another example that I came across on X. Someone posted about an old conversation she had with Claude about not having eaten much for several days because she was sick with a stomach bug. And for weeks afterwards, she said, you know, the chatbot just kept bringing it up, wanting to talk about it as, you know, sort of evidence of disordered eating.
She told it, stop talking about this. And the chatbot said, quote, I'm not going to keep raising it or treat you as a case. I'm only naming it so I'm not quietly behaving differently without telling you. I mean, we don't know, you know, what the sort of intimate details of these conversations were. Clearly, it triggered some sort of safeguard in Claude and was remembering things potentially without the context in which that was inputted into the model.