The Dawn of Dynamic AI: RFT Comes Online, w/ Predibase CEO Dev Rishi, from Inference by Turing Post

**Paige Bailey** (0:00)
This podcast is supported by Google. Hi folks, Paige Bailey here from the Google DeepMind DevRel team. For our developers out there, we know there's a constant trade-off between model intelligence, speed, and cost. Gemini 2.5 Flash aims right at that challenge. It's got the speed you expect from Flash, but with upgraded reasoning power. And crucially, we've added controls like setting thinking budgets, so you can decide how much reasoning to apply, optimizing for latency and costs. So try out gemini2.5flash at aistudio.google.com and let us know what you built.

**Nathan Labenz** (0:31)
Hello, and welcome back to the Cognitive Revolution. Today, I'm excited to share a special crossover episode from Inference by Turing Post, featuring a conversation between host Ksenia Tsai and Dev Rishi, CEO and co-founder of Predibase. I've really been appreciating Ksenia's work recently, both in podcast and newsletter form. She has an exceptional talent for topic selection, repeatedly covering subjects that I've been pondering and wanting to understand more deeply. And her interviewing style is something I'm sure many of you would agree that I could take a lesson from. Short, pointed questions that let the guest do most of the talking. In this episode, Ksenia and Dev tackle one of the few remaining grand challenges on the road to transformative AI, continuous learning. Or as Ksenia's original episode title puts it, when will we train once and learn forever? As we've seen in our recent episode with Ambiance Healthcare, we're already in a world where reinforcement fine-tuning, aka RFT, can turn even modest datasets into dramatic performance improvements, at least on specific, narrow tasks. And already, as Dev explains, some companies are beginning to close the loop, allowing the model to learn not just once from expert curated data, but on an ongoing basis from reward signals including feedback from production users. The implications of this shift from static to dynamic models are highly uncertain, but almost certainly profound. Considering that reinforcement learning has famously delivered superhuman performance on narrow tasks from go playing to protein folding, the upside is undeniably massive. And from an AI safety standpoint, widespread deployment of what Dev calls practical specialized intelligence might offer us a relatively stable AI future. Because by filling economic niches well and cheaply, they would leave less green field for AGI systems to colonize, and thus partially, though not entirely, undercut the economic rationale for continued hyperscaling. For more on that line of thinking, I recommend Eric Drexler's Reframing Superintelligence.
On the other hand, reinforcement learning remains unwieldy, issues of reward hacking loom large, and as we've seen from recent research like the Emergent Misalignment Project, to which I was privileged to make a very minor contribution, out-of-domain behavior can be shockingly problematic. And plus, we still have precious little insight into the dynamics of a world full of continuously evolving specialist AIs. In any case, what makes this conversation particularly valuable is Dev's grounded perspective from the cutting edge of enterprise AI deployment. While so many discussions of online learning remain theoretical, Predibase has been shipping these systems to real customers in healthcare and finance, and has real insight into what works and what challenges remain. Beyond this episode, I also recommend Cassini's conversation with Pinecones CEO Eto Liberty, called When Will We Give AI True Memory? You can find that on the Turing Post YouTube channel by searching for Inference by Turing Post in your favorite podcast app, or by visiting the website turingpost.com, where you can also sign up for the newsletter. Now, I hope you enjoy this visionary but practical preview of continuously learning AI systems with Dev Rishi of Predibase and Cassini Yasei, host of Inference by Turing Post.

**Dev Rishi** (3:46)
We aren't going to live in a world where one model rules at all, but it's incredible at the rate of innovation that we've seen in open source. The world that I see tends to be like, rather than artificial general thought, it's like practical specialized intelligence. The pace here is truly that you will have a breakthrough on expectation about every week.

**Ksenia Tsai** (4:10)
Hello, Devred. Thank you so much for joining me today.

**Dev Rishi** (4:12)
Of course, happy to be here, and thanks for having me.

**Ksenia Tsai** (4:14)
Well, let's start with a big picture, if you can draw me a big picture. When will we train once and learn forever?

**Dev Rishi** (4:22)
It's a great question. I think that world is actually here today. Most of the time when we see customers using models in production, they're taking a model someone else has done 99% of the heavy lifting on, and then they're doing a last mile 1% customization. The trend that I think that's going to go towards what you said, which is like train once and then learn forever, is going to be a shift though where people stop using a static model, this one model someone else trained, and instead have a pipeline that allows them to improve the model continuously while it's in production. We've started to see some of our early customers already put these types of pipelines in practice, and that's what I'm most excited towards being able to build towards as well.