Nate Soares

Alexandra and Andrea Botez (0:00)

What probability would you give to AI causing human extinction?

Nate Soares (0:03)

Exact percentages? Starting to look like we don't have 50 years on the clock. Currently, I would consider us lucky if we had 20, but do we even have two? We know how to make things smart. We don't know how to make those things care about anything in particular. If you have smart things pushing their technology and their civilization, almost every direction leads to the humans dying.

Alexandra and Andrea Botez (0:24)

What does the happy ending look like?

We are closer than ever to creating super intelligence, machine intelligence that is smarter than all humans combined. Companies and nations are competing against each other, pushing acceleration even faster. AI is not there yet, but it is rapidly improving at a pace that is hard to predict. In early 2023, hundreds of AI scientists signed an open letter saying that mitigating the risk of extinction from AI should be a global priority. My guest today, Nate Soares, was one of them. Nate has spent almost a decade focusing on how to make advanced AI actually benefit humanity. Together with Eliza Yudekowsky, he wrote a book titled If Anyone Builds It, Everyone Dies. If we create superintelligence without solving alignment, everyone will die. But this conversation is not about doom. Humanity only gets one shot at this. And if we get it right, I'm really optimistic about the future. Before we get into the heavy stuff, let's start at the beginning. You left Google in 2014 to work full-time as an AI safety researcher. You were working on alignment, which means trying to get advanced AI systems to reliably do what humans want. And this was at a time where AI was barely working, nobody was taking alignment seriously. What did you see so early on that made you take it seriously?

Nate Soares (1:45)

I think a lot of it was just arguments that I thought were pretty good about how humans were trying to automate the process of scientific and technological development. They were making some progress, there wasn't a ton yet, but you know, humans have been trying to build AI since I think the Dartmouth Conference in 1954 It was pretty clear to me that we weren't ready and we weren't on track to being ready. Even back in 2014 where a lot of people said, you know, we have decades, maybe centuries before this stuff really starts working. Even then, it seemed like the sort of problem we should get started on.

Alexandra and Andrea Botez (2:23)

Is it fair to say that AI advanced much faster in the last 10 years than anybody expected at the time?

Nate Soares (2:29)

I mean, probably not literally anybody. There's always people who are like, my project is going to become like a real strong AI in six months. But most people, I think, didn't see the current AI tech coming and didn't see it coming in this particular way. For a long time, there was a saying in AI that things that are easy for humans are hard for computers and vice versa. Like the computers can multiply really big numbers very quickly, but they can't write an essay. And for a while, it looked like that might last for a really long time. Now, of course, the computers can write essays, but that took a lot of people by surprise.

Alexandra and Andrea Botez (3:09)

And one of the hardest things to do with AI is to figure out the timeline. And in your book, you talk a lot about that and how it's basically impossible to predict. And we could look back, and there's times when the Wright brothers were able to fly much earlier than people predicted. At the same time, we still don't have flying cars yet. I think a lot of what makes it difficult for people to fear artificial super intelligence is this feeling that it could be so far away. What do you make of that?

Nate Soares (3:38)

I think the flip side of not knowing how to predict things is it could also be really soon. You know, I think a lot of people's impression of AI is chatbots like ChatGPT, because that's when most people started noticing AI. People who have been in the field for longer than that sort of think of AI not just as these chatbots, but as the field that sometimes has a new insight that leads to like, you know, the fastest growing consumer app of all time being made sort of out of nowhere. And they're like, you know, what's the next thing that happens out of nowhere? What's the next insight, the next breakthrough that happens out of nowhere? How far does that push the AIs? Is it hard for people to visualize? Could it still be a handful of insights off such that we have some handful of years? I mean, it's starting to look like we don't have 50 years on the clock. Currently, I would consider us lucky if we had 20 But do we even have two? It's really hard to predict. I think a lot of people see that uncertainty and conclude it means we have a long time, whereas I see that uncertainty and see that we probably are going to need to act and change course while still uncertain, which is hard for humans to do, but I think necessary.