Beware the AI Prophets: Hype, Doom, and Prediction as Tools of Control w/ Carissa Véliz

Alix Dunn (0:02)

Hey there, I'm Alix Dunn, and welcome back to Computer Says Maybe, where I talk to the people pushing back on tech power and fighting for something better.

It is officially October, which means the US midterms and other global elections are right around the corner. So this month, we are looking at politics and the role that AI is playing in these elections. Really through one question, who gets to shape the future? We are going to look at how powerful people use predictions about technology and the future as political tools to make certain things feel inevitable and in turn narrow the realm of the possible.

Then we are going to get into what happens when people stop believing those predictions or stop trusting the institutions making them. How can that mistrust become leverage? And how can local resistance become sustained political power with a new vision for what the future should look like? Today, I'm joined by Carissa Véliz, author of Prophecy, Prediction, Power, and the Fight for the Future, From Ancient Oracles to AI. Carissa makes a compelling argument that predictions made by people in power aren't forecasts. They are orders, self-fulfilling prophecies. So when we hear Big Tech talking about the inevitability of AI, we have to ask ourselves, are they predicting what's actually going to happen? Or are they just telling us what they want to happen and using their power to make it so?

Throughout the month, we'll be in conversation with people tackling these questions on the podcast and in the Maybe Collective. The Collective is a community we recently launched where you can meet co-conspirators, join events with guests from the show, and dig deeper into the conversations we're having every week. We would love to have you join us, so we'll drop a link in the show notes. Now, let's get into my conversation with Carissa.

Carissa Véliz (1:50)

I'm Carissa Véliz. I'm an associate professor at the Institute for Ethics in AI at the University of Oxford, and I'm the author of Privacy is Power, which is a book about the machinery of surveillance and prophecy, which is a book about how the machinery of surveillance feeds the machinery of prediction, and it's about how prediction has been used and abused for all of history.

Alix Dunn (2:11)

Prediction is a command disguised as description.

I'm going to say that one more time for listeners, because I feel like it's such a dense but also interesting sentence. Prediction is a command disguised as a description. Can you explain a little bit what you mean by that?

Carissa Véliz (2:31)

Because we tend to read predictions as scientific enterprises or as statements that have to do with facts, I thought it was very important to emphasize the history of prediction, which is quite questionable.

And one of the things that make history useful is that when you're not lured in by the distractions of the present time and about feeling fear or admiration or just falling into different tricks or trends, and then when you see it with the perspective of history, you see things more clearly. And one of the things that becomes salient when you look at the history of prediction is that there's a lot of power involved.

And that led me to take out my Philosophers' Toolkit and try to analyze prediction from a different point of view, from a more further away point of view. And one of the tools in the Philosophers' Toolkit is, of course, analyzing language. And when you analyze predictions, any kind of prediction, including very simple ones, like if I tell you, Alix, it's going to rain in a couple of hours, you might think that is a statement of fact, like I'm describing the state of the world in the future. And that might be the case in a way if it rains. And if I'm basing my statement on an app that every time it says, there's a 60% chance of raining, it rains 60% of those times, then even when it's validated, it's actually not a description. At best, it's an educated guess.

And if there's reason, if I can see the clouds and my app says it's gonna rain and the way it smells when it's gonna rain, it might be a very good educated guess and it might be very useful. But even then, it is not a description about the world because descriptions about the world are facts and facts belong to the past and the present, never to the future. So that's the first thing. A prediction can never be a fact, even in the most scientific of context. But furthermore, most of the times, we predict with a certain function in mind. So we don't predict just out of curiosity. We're looking for something. We want something. So there's a lot of what Nietzsche called will to power, weaved into predictions. And when I tell you, Alix, it's going to rain in a couple of hours, maybe I'm telling you we shouldn't go out for a walk, or maybe I'm telling you we should take an umbrella.