Ep 94: Applied Compute CEO on the Limits of RL, the New AI Hyperscaler & Why Post-Training Wins Inference

Jacob Effron (0:00)

How do you kind of characterize where we are today?

Yash Patil (0:02)

What we have with RL is we essentially have a hill climbing machine. The hardest part is actually defining the hill to climb, which is why evals I think is like a really important thing for companies to A, focus on building, and B, actually like safeguard pretty carefully. Your employees are not fungible, you would not like be comfortable with them going to another company and doing work there. Same thing with these models.

Jacob Effron (0:29)

Owning your own intelligence and training your own models is incredibly in the AI zeitgeist these days. There's probably no better person to speak with about all these topics than Yash from Applied Compute. Applied Compute is working with a bunch of the most cutting-edge AI applications on a really interesting post-training examples. Yash and I were able to talk about his perspective on a bunch of the top questions around open models today, including when it actually makes sense to post-train these models, how to think about the future of when this stuff makes sense, especially as the labs get more and more data from different domains over time. We also talked about when it makes sense to harness optimize versus actually go and post-train, as well as just get Yash's thoughts on a lot of the really interesting things happening in the ecosystem today. All in all, this is a fascinating conversation with a brilliant mind in the space. I hope folks enjoyed as much as I did. Without further ado, here's Yash.

Well, thanks so much for coming on the podcast.

Yash Patil (1:18)

Thank you for having me. Yeah.

Jacob Effron (1:20)

Been looking forward to this one for a while.

Yash Patil (1:22)

Yeah. Yeah, no, it's been a long time coming.

It's good we can finally do it in person as well.

Jacob Effron (1:27)

Yes, always better in person in our little closet of a studio here.

Yash Patil (1:30)

Yeah, this is awesome. I mean, it's very cozy, built exactly for this setting.

Jacob Effron (1:36)

Yeah, definitely not a repurposed library. Purpose built for podcasts.

Yash Patil (1:41)

Thanks for having me.

Jacob Effron (1:42)

Of course. Well, I feel like there's a bunch of places we could start.

Where I thought would be most interesting for our listeners is just owning your own intelligence feels like it's in the zeitgeist. I don't know when the tipping point happened three or four months ago, but it is certainly in the conversation. I feel like there's been a bunch of different versions articulated of the risks of depending on Frontier Labs. There's been people saying, you'll never know when they'll pull the APIs away, when they decide something's a safety risk and they don't give you access. There's been, they'll build products and compete with you. They'll be, if you use their models, they'll train on your data. There's all sorts of different arguments. I'm curious, when you think about the case for this, which of these arguments like resonate for you? Which of them do you think maybe are a little overblown?

Yash Patil (2:24)

Yeah, yeah, so, you know, I, as you know, my background is I actually worked at OpenAI. Yeah, you've seen it from the inside. And I think like the risks, or not even risks, just the things that I think about when it comes to owning your intelligence, aren't the ones that are like, presume that the labs are malicious or something like that, where they're stealing your data or whatnot. And I think like these companies are made up of great people, their contractual obligations, there's a lot of engineering that goes in protecting data and whatnot. But I do think the idea of owning your intelligence really is about flexibility and control. And so you said like, oh, I don't know when this own your intelligence narrative popped up. I think it really popped up when the open model started getting good, and you could actually go and do things with them. But the control you get, it can actually be a lot deeper than that, right? Like where these models run, how you deploy them, what you optimize them, whether it's cost or latency or specific domains, the more control you get, that's what we're talking about when we say own your intelligence. So if you want to own your intelligence and you want to be able to train these models to do tasks on your data, you actually need to have access to the weights. You need to have the infrastructure to post-train them, to inference them, to route between them. I do think there are risks about being super dependent on a single model vendor.