Beyond Bigger Models: Recursion As The Next Scaling Law In AI

**Ankit Gupta** (0:00)
Welcome back to another episode of Decoded. Today, I'm back with YC Visiting Partner Francois Chaubard to talk about one of the most interesting recent trends in AI research, recursion. Specifically, we're gonna talk about how we can improve a model's reasoning performance by using recursion at inference time, rather than by just making the model bigger and bigger. There were two papers that made the power of this approach really clear in 2025 One on Hierarchical Reasoning Models, or HRM, and another on Tiny Recursive Models, TRM.
Francois, thanks for joining us. Can you tell us a little bit about these two models and what was so interesting about them?

**Francois Chaubard** (0:42)
Sure.
I guess to set up a little bit of a foundation, you already did an amazing lecture on RNNs and LLMs in one of the previous videos, so I won't overdo it, but just to give the cliff notes, an RNN is just a model that you recursively call again and again and again on itself. And we were very much in the belief that this was required to get to AGI peak RNN uses probably until 2016 with the Alex Graves NeurIPS keynote, which is just fantastic and all his adaptive compute time work.

**Ankit Gupta** (1:15)
So this is about 10 years ago, people were working on these models. This was in the era of LSDMs and LSDMs with attention.

**Francois Chaubard** (1:21)
Yeah, and depending on which professors you talked to before attention was invented.

**Ankit Gupta** (1:25)
Yes, yes, totally, yeah.

**Francois Chaubard** (1:29)
And I think what really was the limiting step on RNNs in general was this thing called backprop through time, where you have to, you roll out the model, and then to update the weights, you need to approximate the gradient, and you step back, back, back, and you keep rolling out. And as the model gets bigger and bigger, and as you roll out for more and more steps, then you have all these accumulation of errors, and the gradient gets noisier and noisier, and then it just kind of stops to work.

**Ankit Gupta** (1:54)
So you have these vanishing or exploding gradient problems, because if you have an input with 20 steps, you're multiplying these matrices 20 times, and that causes training stability.

**Francois Chaubard** (2:01)
And we're talking about doing context length of a million or a billion. And so it's not even just 20, it's a billion. And even worse, you have to retain the activations at every single step. And so if this were happening in your brain, you would need a million copies of your brain at every single activation, so that I can back prop through it. There's tricks around this that you can do, and you can do a gradient checkpointing and things like that to reduce that issue, but then you're just trading off memory for wall clock time and compute.

**Ankit Gupta** (2:29)
Right, so now if you contrast that with LLMs, the ones that people are widely using, these, while at face value they appear to be similar, at training time they're doing basically this one-shot feedforward process for every input. The LLM, the transformer block, can take all of the inputs in parallel. It's not actually iteratively going over them one at a time at train time, so you don't have this needing to store tons of activations problem and this giant vanishing gradients problem with them.

**Francois Chaubard** (2:55)
Yeah, exactly. It's actually all happening in time in one shot, magically. That was the trill or lower triangle trick that happens, this causal mask that occurs. So you actually do all time steps in one shot, and you forward pass a feedforward model on all time steps in one shot, and you backwards in one shot. It's amazing for train time in terms of wall clock.
It requires a lot of flops and it still requires a lot of the memory. You still need it there, but you don't have the vanishing gradient issue.
And what you actually paid for that you have to give up is this latent reasoning thing and this compression in the time direction. There is no compression in LMs. Every single decode that I do, I still have to retain the entire Shakespeare novel just to decode a little bit. And in RNNs, you don't have to do that. It's all compressed in this hidden state that you roll out.

**Ankit Gupta** (3:46)
Okay. So let's talk about that in a little bit more detail.
You refer to this inherent reasoning ability. Many people think about LLMs as doing reasoning, and we're going to talk about that a little bit later. But help me understand where you see the biggest limitations in LLMs reasoning ability is in terms of what the model does in an actual forward pass.