Why AI for Good Fails and How Leaders Can Fix It

Ben Green (0:00)

We need to develop an entirely new direction for developing AI systems that actually work for the betterment of humanity. We don't have to choose between innovation and no innovation. We can choose innovation that's just a different type of innovation than what the big AI labs and tech companies are pursuing.

Cindi Howson (0:29)

Welcome to The Data & AI Chief. I'm your host, Cindi Howson. We want data and AI to make things better, whether in our businesses or society at large. But even well-designed algorithms can fail, potentially hurting the people that they're meant to help.

Our guest today has spent his career figuring out why. Dr. Ben Green is an Assistant Professor of Information and Public Policy at the University of Michigan. He's also a former data scientist for the city of Boston and the author of two books including his latest, Algorithmic Realism.

In this episode, you'll learn what stops data from being used for good, and why AI companies should not regulate themselves, and what we risk losing when we outsource too much of our thinking to AI. Ben, welcome to The Data & AI Chief.

Ben Green (1:29)

Thank you so much. I'm excited to be here.

Cindi Howson (1:32)

I'm assuming you're joining from Michigan, but maybe where as our listeners travel around the world vicariously through our guests.

Ben Green (1:41)

Yes, I am in Ann Arbor, Michigan, where it is a beautiful sunny fall day.

Cindi Howson (1:47)

Great. I guess I should say, go Wolverines. Are you a football fan or are your loyalties more back in Boston?

Ben Green (1:56)

It's both. My professional sports loyalties are fully grounded in Boston sports teams but for college sports, I never really had a team, so now definitely Michigan, those are my teams for all the college sports.

Cindi Howson (2:12)

Okay. Good. Well, it's great football weather, that's for sure. Then when we think about data for good or data for bad, as one of my former colleagues at Gartner, he pushed me to say, just call it data for evil. Data for evil. I'm like, okay, that's really harsh. But a lot of times we have good intentions and disastrous outcomes.

What's helpful to understand so that we can prevent these disasters in the future?

Ben Green (2:42)

Well, I think one of the first things to understand is that the ways in which data scientists and AI developers tend to judge whether a system is good or bad, has a quite tenuous relationship with what the reality of the impacts that the system will create down the road if it's implemented. When we think about data for good going wrong, it's not so much that the data scientists necessarily did a terrible job or didn't understand statistics, it's that their understanding of what makes for a good algorithm is actually quite distinct from what actually leads an algorithm to have socially beneficial impacts in practice.

Cindi Howson (3:26)

Yeah. You talk a lot about that in your book, that data scientists are often measured on how well did the algorithm perform? Did it have this degree of accuracy rather than looking at what was the purpose of the algorithm to begin with? But give us a specific example, make this real.

Ben Green (3:49)

So maybe one example that I'll pull on is about how AI is used in education, actually, because I think that highlights this challenge pretty neatly, where all of these developers are creating AI chatbots and systems and are trying to make these as powerful as possible and are showing that these systems can pass various benchmark tests, and they have all of these capabilities. Then there's this assumption that, well, obviously if you give this to students, it will help them learn because these systems have so much information at their disposal and it can be personalized. But when you actually look at the students using these systems in practice, what you find is that it tends to make their learning and critical thinking worse.

Students become reliant on these systems, they don't engage as deeply in the material. When you look at empirical data, you might find, okay, students are doing a great job at their homework and their take-home assignments when they can use AI. But the second they are put in a situation like an exam where they don't have the AI available, their scores plummet relative to students who didn't have AI available to them. That's just an example of beyond questions of what the goals of a system should be, even taking those goals relatively fixed, just developing a system that looks powerful and capable on a technical level, often doesn't mean that you're actually going to augment the performance or the abilities of people, which is the ultimate end goal of these projects.