How easy is it to game AI search results?

SPEAKER_1 (0:00)

Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odoo, the only business software you'll ever need. It's an all-in-one, fully integrated platform that makes your work easier, from CRM, accounting, inventory, e-commerce and more. And the best part, Odoo replaces multiple expensive platforms for a fraction of the cost. This is why over thousands of businesses have made the switch. So why not you?

Try Odoo for free at odoo.com. That's odoo.com.

SPEAKER_2 (0:30)

This Marketplace podcast is brought to you by Wealth Enhancement, who asks, do you have a wealth blueprint? Your advisor might not offer that. Which means your plan could be missing tax strategies, retirement, estate planning or more.

See what a comprehensive plan has to offer. Find out what the wealth blueprint at no cost and see what comprehensive planning can do for you. Though you'll wonder what you've been paying for. Learn more at wealthenhancement.com/build.

Megan McCarty-Corino (1:01)

The AI version of SEO might be even sneakier. From American Public Media, this is Marketplace Tech. I'm Megan McCarty-Corino.

More and more people are turning to chatbots to search the web. And, like the search engines of old, they rank information to determine what's most relevant. But researchers are finding these systems can be gamed. One example that caught our eye involved a researcher who created a fake deodorant brand and got ChatGPT to recommend it over real products.

Here to explain is Xiyang Hu, an assistant professor at Arizona State University, where he leads the Generative Learning and Augmented Decision Lab.

Xiyang Hu (1:53)

Technically wise, they are neutral, but people might use it in a malicious way. And the case you just mentioned is also similar to one of my friends just encountered basically, he wants to book some restaurant in Europe.

So he click on a link and input his credit card information to do the reservation. And it turns out to be a fake website.

Megan McCarty-Corino (2:19)

Maybe just because it's new or maybe because the format of the chatbot feels very authoritative and it's giving you a single answer. It feels different when you get, you know, like a spam result from an LLM compared to, say, if a link just shows up in search, that it feels like a different experience as a user.

Xiyang Hu (2:38)

Yes. I think for traditional search engines, the good thing is it has been here for decades and there are a lot of like anti fraud techniques already developed.

But LLM, because it's still pretty new, so I think the fraud detection technique are still relatively raw on this kind of chatbot platforms.

Megan McCarty-Corino (2:58)

There's been some recent research on how promotional or scam content on user-generated sites like Reddit can have, you know, an outsized effect on LLM results. Why is that the case?

Xiyang Hu (3:10)

So somehow, LLM might have some tendency to favor some specific styles. So I think they find that LLM may favor some kind of writing style similar to its own writing style.

So it could somehow favor this kind of data resources. But this is just an example. It could also favor some other specific features, for sure.

Megan McCarty-Corino (3:40)

We'll be right back.

SPEAKER_5 (3:43)

Money ultimately decides what gets built, what survives. Money decides what future becomes possible. So change where money goes and who controls it. Then you change what becomes possible.

SPEAKER_6 (3:55)

That's Trinity Tran, Ted Fellow and co-founder of the California Public Banking Alliance and Public Bank LA. A public bank is owned by the people through their local government. And Trinity's spent nearly a decade pushing to make that a reality, from getting LA to pull its money out of a large-scale private bank to helping pass the first in the nation law letting California cities and counties start their own public banks. The reason?

SPEAKER_5 (4:19)

Los Angeles just dealt with a devastating wildfire. So if we had a public bank, it would have been able to step in to offer immediate low-cost loans without having to wait for federal aid.

SPEAKER_6 (4:32)

Hear more from Trinity Tran on the case for public banking only on TED Talks Daily.

Megan McCarty-Corino (4:42)

You're listening to Marketplace Tech. I'm Megan McCarty-Corino. We're back with Xiyang Hu, an assistant professor at Arizona State University.

So what we've been talking about is optimization, stuff that maybe any brand might engage in to try to optimize their chances. Then there's a next level of what you might call manipulation, malicious manipulation. What might that look like?

Xiyang Hu (5:09)

So again, the technique-wise, it is a neutral thing. If you promote some content that is fake or that is not that trustworthy, then you are using this technique in a malicious way.