The AI Coding Factory

**Wix** (0:04)
Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel, and I'm joined by my co-host, Wix, founder of SmallAI.

**Alessio** (0:12)
Hey, and today we're very blessed to have both founders of Factory AI. Welcome.

**Matan Grinberg** (0:16)
Thank you for having us.

**Eno Reyes** (0:17)
Yeah, thank you.

**Alessio** (0:18)
Matan and Eno. My favorite story about the founding of Factory is that you met at the Lying Chain Hackathon, and I'm very annoyed because I was at that hackathon, and I didn't start a company. I didn't meet my co-founder. Maybe you want to quickly sort of retell that little anecdote because I think it's always very fun.

**Matan Grinberg** (0:35)
Yeah, both Eno and myself went to Princeton for undergrad. And what's really funny is retrospectively, we had like 150 mutual friends, but somehow never had a one-on-one conversation. If you pulled us aside and asked us about the other, we probably knew vaguely what they did, what they were up to, but never had a one-on-one conversation. And then at this Lying Chain Hackathon, we're walking around and catch a glimpse of each other out of the corner of our eye, go up, have a conversation, and very quickly just gets into cogeneration. And this was like back in 2023 when cogeneration was all about baby AGI and auto GPT. Like that was like the big focus point there. And both were speaking about it, both were very obsessed with it. And I like to say it was intellectual love at first sight because basically every day since then, we've been obsessively talking to each other about AI for software development.

**Alessio** (1:28)
If I recall that Lanchain Hackathon wasn't about cogeneration, how do you sort of get find the idea maze to factory?

**Eno Reyes** (1:34)
Yeah, basically, I think that we both came at it from slightly different angles. I was at Hugging Face working primarily on advising, like CTOs and AI leaders at Hugging Face's customers, guiding them towards how to think about research strategy, how to think about what models might pop up and, in particular, we had a lot of people asking about code and code models in the context of we all want to build a fine-tuned version on our codebase. In parallel, I had started to explore building. At the time, the concept of agent wasn't really clearly fleshed out, but imagine basically a while loop that wrote Python code and executed on it for a different domain for finance. On my mind was how not very helpful it felt for finance and how incredibly interesting it felt for software. And then when I met Matan, I believe that he was exploring as well.

**Matan Grinberg** (2:30)
Yeah, that's right. So I was, at the time, I was still doing a PhD at Berkeley, technically in theoretical physics, although for a year at that point, I had really switched over into AI research. And I think the thing that pulled me away from string theory, which I had been doing for like 10 years into AI was really the string theory, and you know, physics and mathematics really makes you appreciate, you know, fundamentalness or things that are very general. And the fact that capability in code is really core to performance on any LLM. And like loosely, the better any LLM is at code, the better it is at any downstream task, even that's like writing poetry. And that fundamental beauty of like how code is just core to the way that machines develop intelligence really kind of nerd sniped me and got me to leave what I was pursuing for 10 years. And that mixed also with the fact that code is one of the very few things, especially at the time, that you could actually validate. And so you could have that agentic loop where the LLM is generating the output and you're actually verifying in ground truth the quality of that output. It just made it extremely exciting to pursue.

**Wix** (3:40)
How did you guys decide that it was time to do it? Because I think maybe if you go back, the technology is like, it's cool at a hackathon, but then as you start to build a company, it's maybe like there's a lot of limitations. How did you maybe face out the start of the company of like, okay, the models are not great today, so let's maybe build a harness around it. So then now the models are getting a lot better. So it's time to like go GA as you're doing now and all of that.

**Matan Grinberg** (4:03)
There's kind of a more quantitative answer and then a more qualitative answer. So the qualitative answer kind of building off of what I said before of, you know, it was intellectual love at first sight. I think it was also one of those things that was kind of just like, if you know, you know, we met and we got along so well. And basically, like the next 72 hours, we didn't sleep. We were just like building together on initial versions of what would become factory. And when something like that happens, I think it's good to just like lean in and not really question it and overanalyze. Yet at the same time, if you do actually go and analyze, I think there are exactly the considerations that you're talking about, which is, yeah, the models at the time, which I think at the time, it was just 3.5, which was out. Certainly, that's not enough to have a fully autonomous engineering agent. But very clearly, if you build that harness or you build that scaffolding around it and bring in the relevant integrations or the sources of information that a human engineer would have, it's very clear how that trajectory would get to the point where more and more of the tasks that a developer would do actually come under that line where you can automate it.