Simulation: the new Scaling Law — Joon Sung Park, Simile AI

**Vibhu** (0:03)
Today, we have Joon in the podcast, excited to kick this one off. Very exciting company. I want to kick off and ask you the question, you know, talk us through the story of your life. How have you gotten here?

**Joon Sung Park** (0:14)
Yeah, for sure. So, really excited to be here. A story of my life. So I was born in Korea, and I lived there for a good 11 years or so of my life. And then my family moved to Boston.
So we moved when I was 11
And my parents were doctors, so they were basically going through their postdoctoral studies. My dad was a surgeon, so he was doing his sabbatical years actually at the Boston Children's Hospital. So I grew up there.
Not too close to tech, actually. I was very much like, you know, music, artsy, painting, like that kind of guy. I actually got into painting a little bit later in high school. But that's what I used to do. And then I grew up mostly in the East Coast after Korea. So I lived a good number of years in New Hampshire. And then I went to college in Pennsylvania. And I got into more of this tech scene in college. So I was originally trained to be an artist. I actually thought that would be my professional career. So it wasn't a hobby. It was actually like, hey, let's make a living out of this. And then gradually I got really interested in this idea of, hey, the greatest artist often creates their own medium.
And the best medium that we had available today was actually in computation. So I decided to go deeper into that. And one thing led to another. And obviously we can go deeper into this. But I decided that research was something that gradually that I got interested in. And here I am.

**Swyx** (1:46)
So there's obviously a lot that you packed into the research components. You had one of the best papers of 2023, which was the Generative Agents paper, commonly known as the Smallville paper.
Feel free to call back to anything else that you mentioned. But most people would have heard of you from this, obviously. Do you have any statistics of how many people have read it? Archive gives you something, right? Some stats.

**Joon Sung Park** (2:10)
Yeah, it's a good question. How many people have read it, I'm actually not sure. I know that we do keep track of the number of citations, which I know is going up quite fast.
Google Scholar has 72,000.

**Vibhu** (2:26)
It made a bigger hit, and it was actually a pretty instrumental paper. It was one that got cited so many times.

**Swyx** (2:34)
It is frequently when people ask, what is the best paper of the year, best people you've read recently, it's this one.

**Vibhu** (2:39)
I thought the memory component was pretty underrated. Very good early memory system, but one of the biggest papers.

**Joon Sung Park** (2:48)
Yeah, so maybe I can talk a little bit about how this particular paper came together.
So when I got into research, it was back in 2020 when I started my PhD program at Stanford. And that was the year when we were about to get GPT 3 to be available. So we already had GPT 2, and you could sense that there's this new class of models that was just becoming available in the market. And the team got very intrigued.
And the general consensus was, is this model actually going to be useful for anything? It's really strange that these models are not trying to do any particular task. But we decided to take a bet. So a large group of scholars at Stanford, and it was actually led by one of my co-founders, Percy Liang, came together.

**Swyx** (3:35)
Who coined foundation models.

**Joon Sung Park** (3:36)
Who coined the term foundation model. We wrote this paper where that term came from called Opportunities and Risks of Foundation Model. And during that process, really the thing that I started to think deeply about was, here is a model that is fundamentally new in our ecosystem. The reason why this was new was it wasn't again trained to do anything in particular, but it was its premise was it could do anything and everything. It was like a stem cell if you were to take a biology analogy. And I got really interested in this idea that, well, if we were to really think about what are the killer applications that this particular technology would enable, what would that be? Many of my colleagues were using this for a simple classification, simple generations. Interesting that these models can do that, but from an interaction perspective, not that interesting. We've known how to do that for many decades. And what we came down to was these models are actually trained on this very broad data from the web. So these are human behavioral data. It's the social media, Wikipedia, all these kinds of data. So if you poke at the right angle, then you could see human behavior that would just pop out. That's actually quite realistic. And we've never seen that before.