Recommender Systems Today and Tomorrow

Kyle Polich (0:06)

Welcome to Data Skeptic, a podcast exploring the methods, use cases, and consequences of recommender systems.

Welcome back to the final installment of Data Skeptic Recommender Systems. Two episodes ago, we looked at how recommender systems work and how you measure them. Last episode, we discussed what they optimized for, and some of the modern machinery behind them. This part contains a few odds and ends, but is mostly going to focus on trust, hacking, privacy, and fairness as they intersect with recommender systems.

In earlier times, these were footnotes in the topic of recommender systems. They might not even get a mention in a survey paper. But keeping a close eye on the publication logs, on the archive, peer-reviewed papers and conferences, the season and the research I did for it convinced me that they are becoming main stage topics. So let's start with trust, because if a recommender system fails you too many times, you're going to lose trust in it. And that trust is hard to earn back. Here's Kunal Mukherjee, an Applied Scientist at Zillow, on why explanation is not a nicety. It's actually a retention strategy.

I am always on the go, and I've got the airline points to prove it. And for the longest time, we would just connect to whatever Wi-Fi was available, and I didn't think too much about it. But think about how much of our lives travel through those connections. Email, work, searches, accounts. When something's free, it often means you are the product. So I do have to wonder about that airport Wi-Fi and those hotel or Airbnb Wi-Fis. So that's where ProtonVPN comes in. I use ProtonVPN to add a layer of privacy to my internal communications, especially when I'm on the go. It encrypts my traffic and helps keep what I'm doing online private. And as an added bonus, if I bump into a website or some content that isn't available where I am geographically, I can just change that virtual location and there we go. And what I really appreciate about Proton is their transparency. They have a strict no-logs policy, and it's been independently verified. Oh, I should have led with this. ProtonVPN is open source. So you don't have to take their word for it. Go check out the code or ask an LLM to do it for you. People, or LLMs, can actually inspect the code, and that's great. You also get fast connections, servers in more than 145 countries, and built-in blocking for ads and trackers. It's easy to get started right now. ProtonVPN is offering our listeners 70% off if you get a two-year plan. That's when you go to protonvpn.com/dataskeptic.

Let's spell it all out this time. That's protonvpn.com/d-a-t-a-s-k-e-p-t-i-c for 70% off your two-year plan. That's protonvpn.com/dataskeptic.

Check them out.

Kunal Mukherjee (3:05)

So, how do you stop the suspicion? Because you would see in Amazon, if you search for, let's say, a product, and Amazon gives you some other unrelated products, you don't see it, you just breeze through them.

So, user span is very limited, and getting the confidence of the user is more important in the beginning. Because if you lose it once, the user might just consider it as useless, even after it becomes relevant. So, I said to my manager that, can we create some kind of an explanation system so that we can give the user an explanation why we recommend it? Something like X number of bedrooms, bathroom. So, the user might be like, oh, okay, so this house has this much bedroom, bathroom.

Kyle Polich (3:50)

So, explanations matter.

But here's the humbling part. When you actually put explanations in front of real people and watch what they do, the results are not always flattering. Here's Rowan Schellingerhout, a PhD student at Maastricht University in the Netherlands. Rowan builds explainable recommenders for the high-stakes world of hiring.

Rowan Schellingerhout (4:11)

So, when we looked into their behavior, because we screen recorded everything and we interviewed them asked to talk aloud while working with the system. So, what we basically found out was that people were looking at the explanations, and then basically using it as a source of information, but not as a reason to come to a certain decision. So, they were looking at the explanation and it said like, I don't know, you studied economics, this is a really good fit for you. And they would just read and be like, oh, this person studied economics. I think that would mean they would be a very good fit for this job. So basically, they would use the explanations as an information source, but then still make their own decision. Like regardless of what the system was saying in the explanation, they would use their own experience as a recruiter or a job seeker, or a company representative to come to a conclusion. So, we actually saw that the difference between the random and the real explanations as a result weren't all that large.