Humans&: Bridging IQ and EQ in Machine Learning with Eric Zelikman

**Sarah Guo** (0:05)
Hi, listeners. Welcome back to No Priors. Today, we're here with Eric Zelikman, previously of Stanford and XAI. We're going to talk about the contributions he's made to research, reasoning, and scaling up RL, as well as his new company, Humans End. Eric, thank you so much for doing this.

**Eric Zelikman** (0:22)
Thank you.

**Sarah Guo** (0:23)
You have had an amazing impact as a researcher, including starting from just your time at Stanford. I want to hear about that, but first background of how you got interested in machine learning at all.

**Eric Zelikman** (0:34)
I guess, going back really far, I've been motivated by this question of like, you have all of these people out there, of all of these things that they're really talented in, all of these things that people are really passionate about, that you have like so much, like, you know, there's just so much talent out there. And I've always been like a little bit disappointed that like, you know, like so much of that talent doesn't get used, just because everyone has like circumstances and like has like these, you know, situations where, you know, they can't actually pursue those things. And so for me, AI has...

**Sarah Guo** (1:09)
All of humanity is not living up to their full potential.

**Eric Zelikman** (1:14)
I mean, the thing I've always been excited about is like, how do you actually build this technology that frees people up to kind of do the things that they are passionate about? Like, how do you basically, you know, allow people to actually focus on those things? You know, originally, I thought of automation as kind of like the most natural way of doing it. Like you automate away the parts that like people kind of don't want to do and that, you know, frees up people to do the things that they do want to do. But I guess I realized like increasingly that that's like, it's actually like pretty complex. You actually have to understand if you want to empower people to do what they want to do, you have to really understand what people actually want to do and building systems that understand kind of people's goals and outcomes is actually really hard.

**Sarah Guo** (2:06)
Did you have like this human centric perspective when you were choosing research problems to work on originally?

**Eric Zelikman** (2:13)
I guess like at the very beginning, I was just like, when I was choosing research problems, I was just interested in like, how do you actually make these things half decent?

**Sarah Guo** (2:21)
Okay, so it's more increased capability at all first.

**Eric Zelikman** (2:26)
I think for me, when I looked at like AI or language models back in like 221 or whatever, I was like, these things aren't very smart. They can't do that much. And there was some early work around there, like that show that like, for example, you could use like chain of thought to like, you know, get models to answer more smartly. But it was still like only like a small step improvement at that time. Like there was still the, you know, the benefit of that was, you know, as much as you can really get with just prompting. And so back then, I was like thinking about, okay, how do you actually make them like half decent at actually solving these harder problems?

**Sarah Guo** (3:05)
Can you give a broad, like we have everything from researcher audience to business person audience here. Can you give a broad intuition for a star?

**Eric Zelikman** (3:15)
I guess the intuition is if you have a model and it's able to solve these like basic, like these like slightly harder questions by thinking about them, then what if you actually teach it like, hey, this solution that you came up with, that got you to the right answer. Good job. Or, you know, if you, or if the model didn't, then you basically like don't reward it. I guess the original version of SART actually had like, or yeah, there were like no, there wasn't a baseline at the time. We compared it to Reinforce, which is this like popular algorithm in, I guess, reinforcement learning, like very simple like policy gradient thing. But yeah, I guess, you know, at the time, it was like a very simple algorithm, just, you know, you iteratively generate solutions. If the solutions get you to the right answer, you learn from them. If they don't, you don't. And then you just kind of keep doing this as the model solves harder and harder problems, and then learns from harder and harder problems.

**Sarah Guo** (4:13)
Did you, at what point in the research, if at all, were you surprised by how well it worked? Or did you have some intuition for this being like something scalable?