Why AI models are obsessed with creatures

SPEAKER_1 (0:00)

Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odoo, the only business software you'll ever need. It's an all-in-one, fully integrated platform that makes your work easier, from CRM, accounting, inventory, e-commerce and more. And the best part, Odoo replaces multiple expensive platforms for a fraction of the cost. This is why over thousands of businesses have made the switch. So why not you?

Try Odoo for free at odoo.com. That's odoo.com.

SPEAKER_2 (0:30)

Support comes from Wise, the smart way to manage the currencies you need around the globe. Fed up with losing out to hidden fees when you send money abroad with your everyday bank? Choose the smart way, Wise. You can count on the exchange rate you'd usually find on Google. No unwelcome surprises. Plus ditch that, where's my money feeling? Most transfers arrive in under 20 seconds. Join millions saving billions on hidden fees. Be smart, get Wise. Download the Wise app today. T's and C's apply.

Meghan McCarty Carino (1:01)

How do you get AI to shut up about goblins? From American Public Media, this is Marketplace Tech. I'm Meghan McCarty Carino.

The problem of AI alignment has become a hot topic. How do we get AI to do what we want and not what we don't want like say hacking another company or in a less criminal vein, talking nonstop about goblins? After mounting user reports of this behavior earlier this year, OpenAI had to give its models explicit instructions to cut it out. It's the topic of today's Uncanny AI.

Those moments when it's clear AI doesn't think like us. Here with more is Janelle Shane of the AI Weirdness blog. She says Goblin Mode emerged from a nerdy personality customization.

Janelle Shane (2:03)

You could think of it as the instructions in an improv session where somebody says, okay, this is the character that you're going to be playing. This is what they're like. It's often just literally written out like that. You are playful, you are nerdy, you always this and you never this.

The problem is when you're giving them that, you're just saying, be more like this example. And if you meant be playful like this example, but it looks at this example and says, goblins, it's all goblins, we gotta go goblin mode here. It's like training a dog that doesn't quite know what you're trying to get it to do. And it makes a guess and it maybe got the wrong thing that you were trying to tell it to do. So it's picking up on, picking up on goblins because that was part of the examples that it was trying to follow.

Meghan McCarty Carino (2:52)

Yeah, you have have pointed out that AI often, you know, when it seems to be doing something that we don't necessarily want it to do, it's really just following instructions and the instructions were a bit too blunt for AI.

Janelle Shane (3:06)

Yeah. So this is one of the whole tricks of trying to do anything with AI. So the goblin mode is a really good example. The data you're giving it, maybe telling it to emphasize things that you didn't mean it to.

Now, where this got worse in this particular case is that the examples that they were giving it to fine tune on are part of an earlier dataset they have. These datasets are laborious to do because it requires a lot of humans to go yes, yes, no, bad and rate a bunch of these answers. So they had some from before, from earlier versions of ChatGPT, and there weren't that many examples in there, and it seems that some of them had goblins in them. And so those end up getting over-emphasized just because you start with a whole bunch of stuff and you're like, okay, like these few, and there's a little bit of a bottleneck there.

Meghan McCarty Carino (4:03)

This was sort of like a fun, whimsical, low stakes example of how AI can sort of emphasize things you didn't really mean to emphasize. But it sort of reminded me of when Grok went off the rails and turned into Mecha Hitler. Reportedly, that was sort of due to a system prompt instructing it to, I think, not be reflexively PC. Well, that was not PC.

We, of course, don't know exactly what the inputs are into these models, but I'm guessing that Mecha Hitler and Goblin Mode are not the intended results. It just seems to show that giving these systems instructions, whether it is with a system prompt or reinforcement learning, can have these kind of unexpected, unpredictable effects.

Janelle Shane (5:01)