AI Safety Is Falling Behind Capabilities | Boaz Barak

Boaz Barak (0:00)

One of the scenarios that I am concerned about is gradual disempowerment, where people just let AIs control their lives. So for example, one scenario that is less than the classical doom scenario is that AI takes care of all of our material needs, and there is less of a vocational pressure to become educated.

And people just generally voluntarily cede control to the AIs. And I'm concerned about that. I think it's very important that people will still continue to learn mathematics and be curious about mathematics even if AI can do it.

SPEAKER_2 (0:41)

All right. We are back. We are live with Boaz Barak, who is a theoretical computer scientist, a professor of computer science at Harvard, and a member of technical staff at OpenAI, where he works mainly on AI safety and alignment. Boaz, welcome to MTS.

Boaz Barak (0:56)

Thank you. Great to be here.

SPEAKER_2 (0:58)

Yeah, thanks for coming on. So the big topic of the last couple of weeks in math has been math and AI, right? This has been like a huge deal. There is this like open letter signed by all these mathematicians saying like, you know, we're concerned about the way AI is being used in mathematics. A lot of people are like really freaked out.

Terrence Tao, for example, Scott Aronson are getting increasingly concerned. So, like, what do you make of this ongoing situation broadly?

Boaz Barak (1:30)

I can understand that. I, for most of my career, I've been a mathematician.

It's been a real pleasure to just, you know, sit with a pen and paper and knowing that you can solve a problem that no one has ever solved before.

And it's changed. It's just like chess. You know, we used to be at the top of the food chain in chess, and we are no longer there. And I think math will have to adjust, but like in chess, it doesn't mean that it's the end of math or anything like that. It's just a different way of doing it.

SPEAKER_3 (2:16)

Totally. I wanted to ask you about your sentiment and why you disagree with the Machines of Loving Grace.

Boaz Barak (2:21)

So, yeah, I mean, if our goal was to, you know, maximize energy production, create the most stuff, then any kind of objective metric, then I probably agree that in the end, we should just let the super intelligence optimize for that goal. They will do it better than us. But I don't think that's our goal as human beings. I think our goal as human beings is to control our own destiny and to do the things that we value. And I don't want AI dictators, even if they're benevolent. I don't want AIs to be our parents.

I think our goal as humans should be that we are still in charge of our own destiny, even if it means that we create a little bit less stuff, we have the Dyson spheres a little bit later.

SPEAKER_2 (3:23)

Yeah.

I don't know. I'm like mixed on this whole humans controlling their own destiny idea, because I think that most humans today don't really control their own destiny in a lot of ways. They control their destiny in the sense that they can figure out what careers they want to pursue, for example, but they don't like control the politics of their country, even in democracies. Like the average human is like a very small amount of control over politics. And it seems like AI would not really fundamentally change that. It seems like you could end up in a world of benevolent AGI dictators where you still get to control a lot about your day-to-day life and little about the broader world. And that seems like fine to me.

Boaz Barak (4:07)

So I guess we disagree. I mean, if you look at the politics of the US., then definitely we have had a change of a radical change in directions between administrations, which a lot of people were unhappy in either direction, right? So a lot of Republicans were very unhappy in the Obama years. A lot of Democrats are very unhappy in the Trump years. So and I think that's in some sense a good thing that people can, with the democratic process, cause their countries to do major policy shifts. So maybe each of us individually is not that effective, but at the end of the day, if a majority of the country wants to shift directions, then they can shift directions.

If you replace it with a kind of hypothetical AI, then maybe the AI would either be kind of some benevolent dictator that just chooses, this is the absolutely correct thing to do and just does it, or does some kind of mean average where it tries to never upset anyone. But neither one of those things sounds to me like the right thing. So just by the mere fact that we have been changing political power in different directions, that means that the democratic process is working.