How Anthropic Schismed From OpenAI | Kevin Roose

Kevin Roose (0:00)

Ultimately, it was not any one specific incident. It was sort of this slow accumulation of things that made Dario and Daniela and the rest of that BATNA crew feel like they just, they couldn't trust Sam, they couldn't trust Greg. This had been a long process where he tried to make it work because he thought, you know, this is the lab that's gonna make it to AGI, and I don't wanna waste time going and setting up my own thing, and does the world really need one more AGI lab?

And so, ultimately, you know, he just reached his breaking point, and I think that was avoidable, but I don't think it would have been easy.

SPEAKER_2 (0:37)

We're live with Kevin Roose, who is a tech journalist, he was at the New York Times for around a decade, he was a columnist, he was the host of the podcast Hard Fork, and now is the co-host of the new podcast, Machine Gods, and he's the author of this new book, The AGI Chronicles, which just came out today. So Kevin, welcome.

Kevin Roose (0:57)

Thanks for having me.

SPEAKER_2 (1:00)

Absolutely. So I think the most interesting thing in this entire book was something that a lot of people were not really talking about, which was this Dario memo called The Path to AGI, where he laid out how we're going to reach AGI.

First, OpenAI would train an LLM using all the text on the internet, then they would expand the context window, train it on multimodal data, and then use RL to improve its capabilities in math and coding, and then use its math and coding capabilities to recursively self-improve. And it's like, wow, that's literally exactly what the AI industry has been doing. Like for bar. And Dario wrote this in 2019 So like, what was he able to see that almost no one else at the time, even people in AI, even people who had read less wrong, what was he able to see that they weren't?

Kevin Roose (1:51)

I mean, I think he had been working on this intuition of his that scaling was the best way by far to improve AI systems. He had this other memo that was published for the first time in the book about the big blob of compute back in 2017 So he'd been iterating on this hypothesis that basically, once you have the right architecture, in this case, the transformer-based large language model, you can develop these scaling laws. And he knew that the scarce information of the time that was not public yet in 2019 was that scaling was working and was working in a predictable way.

And as he put it to me, he looked at this thing that they were building, GPT-2, and he said, we could add eight more zeros to this. Like, we could train this instead of on thousands of dollars of compute, we could train this on millions or billions of dollars of compute. And we could pour all of the data on the Internet into it and then more.

And maybe it would just keep improving steadily. So I think that was the thing that he kind of knew before anyone else by virtue of just being part of the team that was, you know, running the team that was working on scaling.

SPEAKER_2 (3:04)

Well, similarly, you know, AI has been extremely popular this year. It's been popular among those in the know on Twitter or whatever for maybe four years now, four or five years.

But it seems like all of the ingredients were already there. And about ten years ago, we had OpenAI already. It was already founded. It was backed by the most famous tech billionaire on the planet. We had AlphaGo. About ten years ago, we had clear evidence of some LLM scaling stuff. So, like, what took everyone so long to take it seriously? Like, yes, I understand that the formalized versions of the scaling laws weren't public until around 2020

But, like, it just seems like the entire world missed out on perhaps the most important thing for such a long time.

Kevin Roose (3:54)

I think that's fair. I think there were people, even inside the AI industry, I mean, one of the chapters in the book is about Google and how, even though they had invented the transformer and, like, come up with so many of the building blocks of modern AI, even though Ilya and Dario and all these superstars had once worked there together at Google Brain, like, they just could not stop building these sort of smaller, narrower, single-purpose models that was just part of their DNA. It was like, you have one model for translation, you have another model for recognizing the images in photos, you have another model for doing, like, autocomplete inside Gmail, and the idea that all of this could be done by, like, a single multimodal model was just kind of anathema to them. It was offensive.