Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit

**Alessio Fanelli** (0:10)
Hey everyone, welcome to the Latent Space Podcast.
This is Alessio, Partner and CTO in Residence and Decibel Partners. I'm joined by my co-host, Swix, Writer and Editor of Latent Space.

**swyx** (0:21)
Hey, and today we have Reza Shabani from Replit, Head of AI. Welcome to the studio.

**Reza Shabani** (0:27)
Thank you, thank you for having me.

**swyx** (0:28)
So we try to introduce people's bios so you don't have to repeat yourself, but then also get a personal side of you. You got your PhD in Econ from Berkeley, and then you were a startup founder for a bit, and then you went into systematic equity trading at BlackRock in Wellington. And then something happened and you were now head of AI at Replit.
What should people know about you that might not be apparent on LinkedIn?

**Reza Shabani** (0:50)
One thing that comes up pretty often is whether I know how to code. Yeah, you'd be shocked. A lot of people are kind of like, do you know how to code? When I was talking to Amjad about this role, I'd originally talked to him, I think, about a product role and didn't get it. Then he was like, well, I know you've done a bunch of data and analytics stuff. We need someone to work on that.
And I was like, sure, I'll do it. And he was like, okay, but you might have to know how to code. And I was like, yeah, yeah, I know how to code. So I think that just kind of surprises people coming from an econ background.
People are always kind of like, wait, even when people join Replit, they're like, wait, does this guy actually know how to code? Is he actually technical?

**swyx** (1:29)
Yeah, you did a bunch of number crunching at top financial companies, and it still wasn't obvious.

**Reza Shabani** (1:35)
Yeah. I mean, I think someone like in a software engineering background, because you think of finance and you think of like calling people to get the deal done and that type of thing. No, it's not that. As you know, it's very, very quantitative, especially what I did in finance, very quantitative.

**swyx** (1:49)
Yeah. So we can cover a little bit of that and then go into the Replit journey. So as you know, I was also a quantitative trader on the sell side and the buy side. And yeah, I actually learned Python there.
I wrote my own data pipelines there before Airflow was a thing.
And it was just me running notebooks and not version controlling them. And it was a complete mess, but we were managing a billion dollars on my crappy code. Yeah. What was it like for you?

**Reza Shabani** (2:17)
I guess somewhat similar. I started the journey during grad school, so during my PhD. And my PhD was in economics and it was always on the more data intensive, kind of applied economic side and specifically financial economics. And so what I did for my dissertation, I recorded CNBC, the Financial News Network for 10 hours a day every day, extracted the closed captions from the video files, and then used that to create a second-by-second transcript of CNBC, merged that on with high-frequency trading quote data, and then looked at, you know, went in and did some NLP tagging the company names and then looked at the price response or the change in price and trading volume in the seconds after a company was mentioned. And this was back in 2009 that I was doing this. So before cloud, before a lot of Python, actually, and definitely before any of these packages were available to make this stuff easy. And that's where I had to really learn to code like outside of, you know, any kind of like data programming languages. It's when I had to learn Python, had to learn all of these other skills to work it with data at that scale.
So then, you know, I thought I wanted to do academia. I did terrible on the academic market because everyone looked at my dissertation like, this is cool, but this isn't economics.
And everyone in the computer science department was actually way more interested in it. Like I hung out there more than in the econ department.
And, you know, didn't get a single academic offer, had two offers. I think I only applied to like two industry jobs and got offers from both of them. They saw value in it. One of them was BlackRock and turned it down to do my own startup. And then went crawling back two and a half years later after the startup failed.

**swyx** (4:03)
Something I liked to say was like you're trading Chinese news tickers or something.