Marketing With Generative AI: Harvard Business School’s Ayelet Israeli

**Sam Ransbotham** (0:02)
How can using generative AI help us understand consumer preferences? On today's episode, hear from a professor about her market research study.

**Ayelet Israeli** (0:12)
My name is Ayelet Israeli from Harvard Business School, and you're listening to Me, Myself, and AI.

**Sam Ransbotham** (0:18)
Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI.
I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and Business Strategy guest editor at MIT Sloan Management Review.

**Shervin Khodabandeh** (0:37)
And I'm Shervin Khodabandeh, senior partner with BCG, and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

**Sam Ransbotham** (1:03)
Hi, everyone. Today, Shervin and I are thrilled to be joined by Ayelet Israeli. She's associate professor and co-founder of the Customer Intelligence Lab at the Data, Digital and Design Institute at Harvard Business School.
Ayelet, thanks for taking the time to talk with us. Let's get started.

**Ayelet Israeli** (1:17)
Thank you so much for having me.

**Sam Ransbotham** (1:20)
Often we begin by asking guests their professions.
But what's nice about being a professor is that people kind of have an idea of what that means. But I still think it'd be nice to hear a little bit about your background and bios. So can you take a minute and introduce yourself and tell us what you're interested in?

**SPEAKER_4** (1:36)
Sure.

**Ayelet Israeli** (1:37)
I'm a marketing professor at Harvard Business School. I'm really interested in how we can better leverage data and AI for better outcomes, if it's outcomes for the firms, for customers, for society at large. Some of the work I'm working on is around gen AI and how can firms use that to gain better access to consumer information and preferences.
In other work I do, I think about how we can eliminate algorithmic bias in our decision making.

**Sam Ransbotham** (2:12)
I saw your talk a few months ago about using generative AI, and it really struck me as interesting because lots of people are talking about generative AI.
But we don't have a lot of evidence yet. Evidence is not saying it's not there, but it's just forthcoming. But you're starting to get some evidence through this research that you're doing. What can we do with GPT in generative and market research?

**Ayelet Israeli** (2:35)
Me and two of my colleagues that are at Microsoft, Donald Wei and James Brand, started thinking around can we actually use GPT for market research? The idea was some people have shown that you can replicate very well-known experiments, including the famous Milgram experiment using GPT by just asking it questions. And we were thinking, you know, we work so much as researchers and as practitioners to better understand customer preferences, maybe we can use GPT to actually extract these kind of preferences.
For large language models, the idea is that they will give you the most likely next word. That's how language is produced. And we were thinking maybe if we ask GPT or induce it to make a choice between two things, maybe the response, which is kind of the most likely next word, will actually reflect the most likely responses in the population.
And in that sense, we will essentially query GPT, but get kind of the underlying distribution of preferences that we see in the population. And we started playing around with that idea. We focused on consumer products because we assumed that the data that GPT is aware of is mostly around consumer products, maybe from review websites or things like that, to see can this idea actually work.

**Shervin Khodabandeh** (4:06)
And does it?

**Ayelet Israeli** (4:07)
Kind of.

**Shervin Khodabandeh** (4:08)
That's wonderful. Yeah. So tell us more.

**Ayelet Israeli** (4:11)
Our first rush was like, OK, let's see if it can generate very basic things we expect from economics. Like when the price is higher, does it know to reject an offer? Does it know to make this trade off between price and choice? And we do see kind of a downward sloping demand curve, which is what you would expect to see when we query GPT, you know, thousands of times to get answers. We also see things like, oh, we can tell it something about its income and it reacts to that when it has higher income, it's less price sensitive, which makes sense, is what we expect from people as well. We also see that it can react to information about itself like, oh, your last time you bought in this category, you bought this particular brand, makes it much more likely to pick this brand in the future. Those are our tests of does it actually react in a way that humans would in surveys?