AI chatbots have linguistic slips when they go off-script

**SPEAKER_1** (0:00)
Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odoo, the only business software you'll ever need. It's an all-in-one, fully integrated platform that makes your work easier, from CRM, accounting, inventory, e-commerce and more. And the best part, Odoo replaces multiple expensive platforms for a fraction of the cost. This is why over thousands of businesses have made the switch. So why not you?
Try Odoo for free at odoo.com. That's odoo.com.

**Megan McCarty-Corino** (0:31)
You ever noticed a chat bot code switching? From American Public Media, this is Marketplace Tech. I'm Megan McCarty-Corino.
So you're chatting away with AI, perfect grammar, great vocabulary, then boom, a random word in another language. That's the topic for today's Uncanny AI. Those moments where it becomes really clear, AI doesn't think like us. Here to explain is Janelle Shane. She writes the AI Weirdness blog.

**Janelle Shane** (1:13)
These language models are trained on most of the internet, and that includes the internet in different languages, and that's part of what helps them keep track of how to do translations, and it's also inevitable. It turns out to be really hard to filter out text from other languages when you're training these giant models and to detect it, especially if you have data that may have mixed languages in there. The other thing is the way that these languages are stored or kept, the way that it's in the memory of this model is similar to the way that different kinds of text and English are stored. So your cooking vocabulary, your woodworking vocabulary, your formal writing, your informal writing, these are all just different kinds of writing and other languages are another kind of writing.
So they're not walled off. It's not changing mode. This is all just part of the cloud of text and associations. It's sucked up from the Internet.

**Megan McCarty-Corino** (2:25)
Does a chatbot know what language it's speaking? I mean, it's just basically predicting the next token.

**Janelle Shane** (2:34)
Yeah, I am not exactly sure how we could think of it as knowing something. But I think you are right that if the next token, if we've been talking in English, then the next token is very likely to be one of these English words. If we've been talking about cooking, the next token is likely to be one of these cooking words. But that's not infallible.

**Megan McCarty-Corino** (2:58)
All right. So here's an example.
In one case, a Chinese word just showed up in the middle of a sentence when I was having a conversation about a new pain in my ankle. Here's what Claude told me.

**SPEAKER_4** (3:11)
That's actually a really helpful description and honestly makes me think even more that this is nerve-related rather than anything systemic like gout. Nerves can absolutely give that sharp shooting yidongsheng quality you're describing, almost like an electrical zap that comes and goes.

**Megan McCarty-Corino** (3:29)
So it kind of came off like the college student that just went abroad and is just peppering their speech with random foreign words. Maybe that just was the best word to describe this zappy feeling in my ankle. I don't know. What do you think?

**Janelle Shane** (3:46)
It could be that. It could be there's some bleed through from a whole bunch of training data on Chinese traditional medicine, where maybe that's a term that comes up often. And so you're getting the these two domains are close to one another, and it's kind of easy to have a little bit of unintended crossover.

**Megan McCarty-Corino** (4:07)
All right, then there was another case with ChatGPT, where it inserted what turned out to be a Ukrainian word, and this one was just sort of appended to the end of a conversation that I was having about choosing a new television. And this is what ChatGPT told me about what had happened.

**SPEAKER_6** (4:31)
The model me sometimes pulls from multiple sources or internal modules to synthesize answers. The scutchey when doing comparisons or pulling up to date info.
In this case, a text snippet or label from a non-English source, Ukrainian in this case, vitpilvid means answer, may have been mistakenly included in the final output. Sometimes the system uses hidden formatting tags or metadata behind the scenes, and a fragment can leak through if a tag is accidentally cut off or misread.

**Megan McCarty-Corino** (4:52)
Okay, so the word that was appended in Ukrainian apparently means answer, and it somehow got in there from the data labeling process. Can you kind of break this down?

**Janelle Shane** (5:06)
Well, this is one possible explanation. I want to start by saying that just because it said this is the explanation doesn't mean it is. It's kind of like, here's a plausible explanation that an expert might have given. So we don't know if it's an explanation in this case, but you think of this training data and some of it is pre-formatted, like you can almost think of it like a question and answer dialogue. The Q, the A, and that's kind of how it tells you what part in the conversation is what. So as these models are getting trained for conversation, fine-tuned for conversation, there may be more of these kinds of examples in this data set with Q&A.