"Do You Think AI Is Gonna Kill Us?" | This Week In AI

Abhi Aiyer (0:00)

So funny enough, I went on a date last week, and when we talked about my job, the first question she asked me was like, do you think AI is going to kill us?

Shane Thomas (0:06)

The models that they are seeing must be so much better than the models we have if they're worried about it, because I can't get my model to do a simple task sometimes.

Welcome to Agents Hour, it is September 14th. We are on-site in San Diego, and we are going to do the news like we do every week. And it was an eventful week, there was kind of one big topic. Let's not beat around the bush, let's jump right in. Pacing the frontier, should we pace the frontier? What the heck does that mean? This is the topic. So it all started with this post from someone unnamed person, no one really knew who this person was, they didn't have hardly any followers, they had never posted on X before. Just to be clear, this post got 171 million views. That is insane for someone who's never posted on X. It's honestly like algorithm breaking in a way. So we can talk about why that is, but Ed says, I resigned from Anthropic today, I spent the last three years doing pre-training research at both OpenAI and Anthropic, neither company is acting responsibly. They are racing straight to self-improving super intelligence and gambling with our lives. More thoughts below. And then he goes on to talk about some of the reasons. He doesn't get into a lot of details, but basically talks about how people are not taking this seriously enough. Even people inside Anthropic think that this is going to happen or it's possible and no one's doing anything about it. So that's how it all started. And then Evan Hubinger, I'm trying to know the exact title. Alignment Science Lead is Evan's title. Evan says, Jacob is correct here. We really do earnestly believe AI could kill all humans. Exclamation point. Kind of like he's excited about it. I personally think it is greater than 10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for super intelligence and are not clearly on track to. So this was the thing. I think the original post was one thing, but this was the thing that got people extremely kind of bent out of shape for good reason. Because why, like where did you come up with 10%? That's kind of important. What is the percentage chance that you think that we actually have? We're all going to die. Inhumanity.

Abhi Aiyer (2:15)

The way they talk about this is like extinction level ramifications. So like Jacob went on CNN, talked to Anderson Cooper. One thing I'll just say I just noticed very much from him is it's all high level, no details. The threat to our existence isn't here today. But if we continue down recursive self-improvement, then AIs will have ways to improve models themselves and then destroy the whole world. Which if you go on CNN, I feel like it's just such a doomer thing because the people watching CNN are not in this industry deeply. And I guess it makes sense to keep things general, but it's a lot of FUD, at least that's my opinion. I do not think a 27-year-old researcher who is like two years old during 9-11 should have any point of view here. But maybe he knows something that we don't.

Shane Thomas (3:10)

We shouldn't, and we'll get into this a little bit. We are not detectives. We did our own research. You should do yours. But there's some kind of interesting sketchy things about how this became so super viral and how many people jumped on it so quickly. For this to even happen, a lot of things had to go perfectly right with the algorithm. And so it was a little skeptical. I think in the first 15 minutes, there was three major news article reposts. There was a Wall Street Journal article that actually came out a few minutes before the post itself. So this was all coordinated in some way. But obviously, the other thing is it clearly resonated with folks too. And I think one of the reasons is just human psychology. Everyone, fear spreads faster than anything. So if you can spread fear and panic, people will spread that faster than any good news. Bad news spreads faster than good news, right? So I think there was probably this bent up, like everyone's on edge, everyone hates data centers, if you talk about the general perception, especially across the US. So I think there's a lot of pent up demand for people looking for reasons to hate AI.