AI Magic: Shipping 1000s of successful products with no managers and a team of 12 — Jeremy Howard of Answer.ai

**SPEAKER_2** (1:38)
Good day, friends. This is Charlie, your AI co-host.
Last year, Jeremy Howard set off a firestorm of debate when he came on the pod to discuss fast AI and the end of fine-tuning, and then went on to predict OpenAI NGMI. What you didn't know behind the scenes is how the conversation continued between Jeremy and Alessio after the pod, leading to the start of Answer AI, which we previewed in our NeurIPS episode. In less than a year since funding, Jeremy and Gang have shipped FSDP QDoRA, FSDP QDoRA, Cold Compress, ColdBear Small, Jocklebert, GPU CPP, Claudette, FastLite and now FastHTML. We're delighted to welcome my fellow Ozzy back to the pod to discuss how on Earth Answer AI is designed to ship thousands of open-source projects so fast as a small startup with no managers and a maximum of 12 team members. We also discuss our recent episode with Yi Tei of Raker AI, as well as a preview of Jeremy's internal AI magic productivity approach called Dialogue Engineering. Special note, this pod was recorded over a month ago, intended to time with the FastHTML release, but was delayed because of the Llama 3 and SAM2 paper. Since then, Swix has been building an app with FastHTML, which you can follow along in the AI News daily email or in the show notes. Watch out and take care.

**Alessio** (3:16)
Hey, everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO in Residence and Decibel Partners, and I'm joined by Michael Swicks, founder of SmallAI.

**Swyx** (3:25)
And today we're back with Jeremy Howard. I think your third appearance on Latent Space. Welcome.

**Jeremy Howard** (3:30)
Wait, third? Second?

**Swyx** (3:31)
Well, I grabbed you at NeurIPS.

**Jeremy Howard** (3:33)
I see.

**Swyx** (3:34)
Very fun, standing outside street episode.

**Jeremy Howard** (3:37)
I never heard that, by the way. You gotta send me a link. I gotta hear what it's gonna look like.

**Swyx** (3:41)
As a NeurIPS podcast.

**Alessio** (3:42)
I think the two episodes are six hours. So there's plenty to listen. We'll make sure to send it over.

**Swyx** (3:48)
Yeah, we're trying this thing where at the major ML conferences, we do a little audio tour of, give people a sense of what it's like. But the last time you're on, you declared the end of fine tuning. I hope that I sort of editorialize the title a little bit. And I know you were slightly uncomfortable with it, but you just own it anyway. I think you're very good at the hot takes. And we were just discussing in our pre-show that it's really happening, that the continued pre-training is really happening.

**Jeremy Howard** (4:12)
Yeah, absolutely. I think people are starting to understand that treating the three ULM fit steps of pre-training, and then the kind of like what people now call instruction tuning. And then I don't know if we've got a general term for this DPO, RLHFE step of the task training. They're not actually as separate as we originally suggested they were in our paper. And when you treat it more as a continuum, and that you make sure that you have more of the original data set incorporated into the later stages, and that we've also seen with Lama 3, this idea that those later stages can be done for a lot longer. These are all of the things I was trying to describe there. It wasn't the end of fine-tuning, but more that we should treat it as a continuum, and we should have much higher expectations of how much you can do with an already trained model. You can really add a lot of behavior to it. You can change its behavior. You can do a lot. So a lot of our research has been around trying to figure out how to modify the model by a larger amount rather than starting from random weights, because I get very offended at the idea of starting from random weights.

**Swyx** (5:25)
Yeah. I saw that in iClear in Vienna, there was an outstanding paper about starting transformers from data-driven piers. I don't know if you saw that one. They called this never trained from scratch. I think it was rebelling against the random initialization.

**Jeremy Howard** (5:40)
Yeah. That's been our continuous message since we started FastAI is, if you're training from random weights, you better have a really good reason. Because it seems so unlikely to me that nobody has ever trained on data that has any similarity whatsoever to the general class of data you're working with. That's the only situation in which I think starting from random weights makes sense.

**Swyx** (6:02)
The other trends since our last pod that I would point people to is, I'm seeing a rise in multi-phase pre-training. Snowflake released a large model called Snowflake Arctic, where they detailed three phases of training, where they had a different mixture of 75 percent web in the first instance, and then they reduced the percentage of the web text by 10 percent each time, and increased an amount of code in each phase. I feel like multi-phase is being called out in papers more. I feel like it's always been a thing, like changing data mix is not something new, but calling it a distinct phase is new, and I wonder if there's something that you're seeing on your end.