Ep 84: OpenAI’s Chief Scientist on Continual Learning Hype, RL Beyond Code, & Future Alignment Directions

Jakub Pachocki (0:00)

I definitely agree that continual learning is really the thing. It's really the thing that we're building, but I don't really think this is a problem that's ignored and off the path of what we're doing currently. I think it is what we're working towards.

Jacob Effron (0:09)

What are the other research areas within Alignment that you're paying attention to or that you think are promising?

Jakub Pachocki (0:13)

A lot of the longer-term challenge with Alignment is about generalization. What are the values that the model falls back on?

Jacob Effron (0:20)

What are the things that you need to figure out to be able to really make models work well in some of these other spaces?

Jakub Pachocki (0:25)

I come back to this while I'm here.

Jacob Effron (0:26)

Jakub Pachocki is the chief scientist of OpenAI. I think literally one of the most important people on the planet. Today on Unsupervised Learning, I got to ask him literally everything that I've been thinking about, and I know a bunch of people in the ecosystem have too. We talked a lot about model progress, what's required to make long-running agents work, as well as the really interesting work OpenAI has done in the AI for Science world, and the progress he sees in that over the next years. We talked a lot about how companies should be thinking about model building in this moment, when they should be doing reinforcement learning, how they should be thinking about the evolution of harnesses and the impact that will have. We hit on a lot of his really interesting research, including the work he's done around alignment, the work that OpenAI broadly has done around math competitions. And we also talked about this focusing moment at OpenAI and what it means for the research organization and how he runs his team. Literally just such an awesome opportunity to talk to someone who is driving so much of the change that has revolutionized the space in the world. I hope folks enjoy this wide-ranging conversation as much as I did.

I feel like you are the perfect person to talk to you about all the questions everyone has in the ecosystem. What's happening with model progress? A lot of companies are thinking about how they should be building things based on what's happening with the models. A lot of people at a societal level are thinking about the impact AI is going to have on science and broader society. And you've been at the forefront of the space for pretty much every generation of improvement these past years. And so, really excited to have you on the podcast. I think I'll start with one of the juiciest things you said, which is, four months ago, I think you and the OpenAI team talked about aiming for a system with research level entering capabilities by September of this year. So, coming up, I think that's six months from now, and then a more fully automated AI researcher by March 2028 And so, I guess checking in four months later, how are you feeling about those timelines?

Jakub Pachocki (2:16)

Yeah, I think over the last months, I think the change that really happened is we've seen this explosive growth of coding tools.

Jacob Effron (2:26)

It's an understatement.

Jakub Pachocki (2:28)

Yeah, we definitely really kind of gone to a place in OpenAI where we use codecs for the majority of actual coding. And so, I think for most people, the act of programming has changed quite a bit.

So, I definitely see this as a signal that something here is on track. The other kind of very interesting update over the last few months, to me, has been the progress on the math research capabilities. Also, the results we've seen in physics, in other fields. I think this kind of level of capabilities, the ability to provide insight when combined with ability to access infrastructure, ability to use, maybe more computer test time than something that comes to use currently. And very strong improvement in general intelligence, which I also expect over the next couple of months. Yeah, it's something we're still very much planning for and very focused on.

Jacob Effron (3:29)

And how do you know when you've gotten there? What's a workflow you might look to to say, hey, okay, I think we've got these research intern level capabilities?

Jakub Pachocki (3:38)

The way I would distinguish a research intern from full automated researcher is the span of time that we would have it work mostly autonomously, or the specificity of the task that it has to be given. So I don't expect we'll have systems where you just tell them all go, improve model capability, go solve alignment.