High Agency Pydantic > VC Backed Frameworks — with Jason Liu of Instructor

**Alessio** (0:06)
Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO in Residence and Decibel Partners, and I'm joined by my co-host, Swix, founder of Small AI.

**Swyx** (0:16)
Hello, we're back in the remote studio with Jason Liu from Instructor. Welcome, Jason.

**Jason Liu** (0:20)
Hey there. Thanks for having me.

**Swyx** (0:23)
Jason, you are extremely famous, so I don't know what I'm going to do introducing you, but you're one of the Waterloo Clan. There's like the small cadre of you that's just completely dominating machine learning.
Actually, can you list Waterloo alums that you know are just dominating and crushing it right now?

**Jason Liu** (0:39)
So, John from Rysana is doing his inversion models, right? I know Clive Chen from Waterloo. When I started the data science club, he was one of the guys who were joining in and just hanging out in the room, and now he was at Tesla working with Carpathy, now he's at OpenAI.

**Swyx** (0:56)
He's in my climbing club.

**Jason Liu** (0:57)
Oh, hell yeah. I haven't seen him in like six years now.

**Swyx** (1:01)
To get in the social scene in San Francisco, you have to climb, so both in career and in rocks. So you started the data science club at Waterloo, we can talk about that, but then also spent five years at StitchFix as an MLE.
You pioneered the use of OpenAI's LLMs to increase stylist efficiency. So you must have been like a very, very early user. This was like pretty early on.

**Jason Liu** (1:20)
Yeah, I mean, this was like GBD3. Okay, so we actually were using transformers at StitchFix before the GBD3 model. So we were just using transformers for recommendation systems. At that time, I was very skeptical of transformers. I was like, why do we need all this infrastructure? We can just use like matrix factorization. When GBD2 came out, I fine-tuned my own GBD2 to write like rap lyrics. And I was like, okay, this is cute.
Okay, I got to go back to my real job, right? Like who cares if I can write a rap lyric?
When GBD construct came out, again, I was very much like, why are we using like a post request to review every comment a person leaves? Like we can just use classical models.
So I was very against language models for like the longest time. And then when ChachiPD came out, I basically just wrote a long apology letter to every one of the companies. Like, hey guys, you know, I was very dismissive of some of this technology. I didn't think you would scale well, and I am wrong. This is incredible.
And I immediately just transitioned to go from computer vision, recommendation systems to LLMs. But funny enough, now that we have RAG, we're kind of going back to recommendation systems.

**Swyx** (2:21)
Yeah, speaking of that, I think Alessio is going to bring up the next one.

**Alessio** (2:23)
Yeah, I was going to say we had Bregan Bishop from X on the podcast to overlap StitchFix.

**Jason Liu** (2:28)
Yeah, he was one of my main users of the recommendation framework that I had built out at StitchFix.

**Alessio** (2:32)
Yeah, we talked a lot about REXIS, so it makes sense.

**Swyx** (2:36)
So now I have adopted that line, RAG is REXIS.
And if you're trying to reinvent new concepts, you should study REXIS first because you're going to independently reinvent a lot of concepts. So your system was called Flight. It's a recommendation framework with over 80% adoption servicing 350 million requests every day. Wasn't there something existing at StitchFix? Why did you have to write one from scratch?

**Jason Liu** (2:56)
No, so I think because at StitchFix, a lot of the machine learning engineers and data scientists were writing production code, sort of every team's systems were very bespoke. It's like this team only needs to do like real-time recommendations with small data so they just have like a fast API app with some like pandas code. This other team has to do a lot more data so they have some kind of like spark job that does some batch ETL that does a recommendation, right?
And so what happens is each team writes their code differently and I have to come in and like refactor their code. And I was like, oh man, I'm refactoring four different code bases, four different times. Wouldn't it be better if all of the code quality was my fault? All right, let me just write this framework, force everyone else to use it.
And now one person can maintain five different systems rather than five teams having their own bespoke system. And so it was really a need of just sort of standardizing everything. And then once you do that, you can do observability across the entire pipeline and make large sweeping improvements in this infrastructure, right? If we notice that something is slow, we can detect it on the operator layer, just, hey, hey, like this team, you guys are doing this operation, it's lowering our latency by like 30%. If you just optimize your Python code here, we can probably make an extra million dollars. So let's jump on a call and figure this out. And then a lot of it was doing all this observability work to figure out what the heck is going on and optimize this system from not only just a code perspective, sort of like harassing the Oregon saying like, we need to add cash in here, we're doing duplicated work here.