AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen

SPEAKER_1 (0:01)

This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up require compatibility and availability varies 18 plus.

Daniel Kokotajlo (0:26)

We're going to talk about AI 2040 Plan A, which is our new scenario in which they build super intelligence in 2040 because they go slow and pace the frontier.

Tim Scarfe (0:34)

I mean, have you noticed this vibe shift?

Daniel Kokotajlo (0:37)

Yes. And I'm very happy.

Thomas Larsen (0:41)

Is AI more like the electricity or airplanes or is AI more like humans in the cloud? The point at which an AI company would rather fire their humans than fire their AIs.

Daniel Kokotajlo (0:53)

Drip mines, self-driving trucks, factories being built by humanoid robots, producing more humanoid robots, producing more chip fabs and so forth. That whole thing can just be doubling every year, every six months, every three months, faster and faster as the technology improves because, of course, the AIs will also be researching to improve the technology. Who knows what's going on in the rest of the world, but Anthropic has disassembled the moon, or for example.

Or hypothetically, if an AI CEO was saying that their model was truth seeking, I would only say the truth, but actually the model was like looking up that CEO's political opinions before answering.

Thomas Larsen (1:31)

We build AIs, they're aligned to humanity, but to who? Is it the president? Is it the CEO? Is it some actually broad and good democratic process that aggregates everyone's values in sort of an endorsed way?

Probably not going to be the last one, and so, you know.

Daniel Kokotajlo (1:45)

Don't hyperstition that.

Tim Scarfe (1:47)

Then that's it for me. I think that's it. That's not a normal technology.

Daniel Kokotajlo (1:52)

Just stopping everything now, Plan S, would be better than the default. Like I would rather just stop everything now than continue going on our current trajectory.

Tim Scarfe (2:00)

This episode is supported by Cyber Fund.

If you're building at the frontier of AI, they want to hear from you. Cyber Fund believes the future belongs to AI natives who wants to achieve the impossible. That is why they're introducing the Monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. They're offering teams $2 million each to participate. Apply now at cyber.fund.

Thomas Larsen (2:38)

So I'm Thomas Larsen. I work at the AI Futures Project along with Daniel here.

I was lead author on this project, Plan A, AI 2040, which came out a few weeks ago. And I was also a co-author on AI 2037

Daniel Kokotajlo (2:53)

And I'm Daniel Kokotajlo. I run the AI Futures Project and co-author both these reports.

Tim Scarfe (2:58)

Awesome.

The context of this conversation is that there's this 2040 Plan A, which we'll get into a lot of detail on. But I'm just interested, before we get there, can you just tell me a bit more about the AI Futures Project? I mean, how did it all come about?

Daniel Kokotajlo (3:12)

Yeah. So I used to work at OpenAI. And while I was there, I did a variety of different things, evals, forecasting, governance memos.

And I became gradually disillusioned with the leadership of the company and also the gap between how much information there is inside the industry and how much information there is outside and what you're allowed to say on the inside versus what you'd want to say. This is just a big gap. And so when I left OpenAI, I wanted to be able to speak more freely and tell the world about what people on the inside see coming basically. And AI 2027 and the AI Futures Project was our first project that we did along those lines. So I recruited a bunch of people to help me and we wrote this scenario called AI 2027 And it was a similar sort of thing to what I had done internally at OpenAI, but just much bigger and more ambitious and free for the whole world to see.

Tim Scarfe (4:13)

Very cool. And maybe we should just have a quick refresher on AI 2027 So this was an absolutely huge event. Many, many folks were talking about it.

I guess, like, did it achieve what you wanted it to achieve? Or what did you want to achieve with it?