An Industry Benchmark for Data Fairness: Sony’s Alice Xiang

**SPEAKER_1** (0:03)
Hi, listeners. We're running a short survey to learn more about our audience so that we can continue to bring you a podcast you find helpful. If you have a moment, please take the survey at mitsmr.com/podcastsurvey. You'll receive a complimentary download of MIT SMR's executive guide, How to Manage the Value of Generative AI. Please take the survey this month at mitsmr.com/podcastsurvey. We'll put that link in the show notes and thank you for your help.
Why did one large company decide to create a data fairness tool that's free and publicly available? Find out on today's episode.

**Alice Xiang** (0:50)
I'm Alice Xiang from Sony, and you're listening to Me, Myself, and AI.

**Sam Ransbotham** (0:55)
Welcome to Me, Myself, and AI, a podcast from MIT Sloan Management Review, exploring the future of artificial intelligence. I'm Sam Ransbotham, professor of analytics at Boston College. I've been researching data, analytics, and AI at MIT SMR since 2014, with research articles, annual industry reports, case studies, and now 12 seasons of podcast episodes. On each episode, corporate leaders, cutting-edge researchers, and AI policy makers join us to break down what separates AI hype from AI success.
Hey, listeners. Thanks again to everyone for joining us. Today, I'm talking with Alice Xiang. She's the global head of AI governance at Sony and lead research scientist for AI ethics at Sony AI. She leads a team that guides the establishment of AI governance policies and frameworks across all of Sony's business units. She's been a research scientist and a whole bunch more. Alice, thanks for joining us.

**Alice Xiang** (1:59)
Thank you so much for having me.

**Sam Ransbotham** (2:01)
Hey, to start, we first talked a few years ago when you were at the partnership on AI, but I'm curious what you're up to now. Can you tell us about Sony's work on responsible AI ethics and governance?

**Alice Xiang** (2:11)
Yeah, sure. Sony is a large multinational company headquartered in Japan with a diverse array of businesses around creative entertainment and technology. We have operating companies focused on music, motion pictures, video games, and electronics. AI became an early focus of ours back in 2018 when we first set up our AI ethics guidelines. As a technology company, we wanted to ensure that this new and emerging technology was being used responsibly across our business units. And indeed, since then, it's only grown in importance for our company. I have two hats on at Sony. One is as global head of AI governance, where over the past several years, we were one of the early companies to start investing in AI ethics. And when I joined, I established our AI ethics office and our processes in terms of how different AI uses and AI integration into products and services is evaluated for responsible AI. Now we're at the point of not just thinking about AI ethics, but also AI governance. So how do we establish these frameworks for ensuring the responsible evaluation of these technologies that are increasingly being integrated in every aspect of business? And that's kind of one hat that I have on the kind of policymaking, guidance setting, so on and so forth. And then the other is leading our AI ethics research team within Sony AI. A lot of the work of my team over the past several years has been looking at what are some of the fundamental gaps and barriers that practitioners face in terms of being able to develop responsible AI in practice. And one of the major areas there is lack of ethically sourced data. Even for pretty basic things like being able to check for bias and models, there's not really great fairness evaluation data sets in many areas like human-centric computer vision. So we've been doing a lot of work there and that's recently culminated in publication in Nature of our Fair Human Centric Image Benchmark, also known as Phoebe. And so we really hope that our work can help enable the broader community, both within Sony and outside, to be able to move towards more trustworthy and responsible AI development.

**Sam Ransbotham** (4:36)
Yeah, I got pretty excited when I saw the Phoebe work. I think it's pretty interesting. It's definitely a problem and I'm glad, glad there are a few people working on this. Give us some details, like what exactly is involved in Phoebe and what are the pieces and how do I use it tomorrow if I want to?

**Alice Xiang** (4:52)
It's so interesting because when I first got into this AI ethics space around computer vision, I kind of assumed that a lot of my role would be helping people on how exactly to do fairness assessments. Like it's a pretty difficult area in terms of how do you measure fairness, what does fairness mean, what do you do when you have biases. But I realized actually the biggest initial barrier folks have is even just being able to evaluate for bias and so theoretically what you want there is you want a data set that's been ethically sourced. So that means there's been appropriate consent and compensation and sourcing throughout the process so everyone who's participated in that data collection process has been appropriately compensated and has consented to their data being used and has control over how that data is being used. And then also for bias evaluation in particular, you want to have a very diverse, ideally globally diverse data set because you don't want to be checking for bias, but all of your subjects have light skin. For example, in that case you really wouldn't be able to tell whether your model performs well on darker skin tones.