NeurIPS 2023 Recap — Top Startups

**Swix** (0:06)
Hello, hello, this is Swix, back again with part two of our NeurIPS coverage. This time, we're going to cover startups.
And it's a special episode because this is the last episode of 2023 We are definitely looking back at the year with rose-colored glasses. This has been a fantastic year. We only started this podcast in February, and it's grown so much. Thanks to all of you who've listened and give feedback and shared it with your friends.
And we actually managed to invite a few of our former guests back on the pod together with some new friends. And probably some new voices that you're going to be hearing next year. So this is not a hard hitting interview series. You know, it's not going to, it's not that kind of interview. It's not that kind of podcast where we try to go too deep. We, today we're just going to go broad and we're just going to check it on a bunch of startups that we like and monitor. And we're present at NeurIPS. So first up is Jon Frankle of MosaicML. We last talked to him in May for the MPT 7B episode. That's episode 13
And I have to say that was one of the best performing episodes of the whole year. So you're welcome to go back and listen to that if you missed it. And since then, they were bought by Databricks for $1.3 billion. And actually during the interview, they were in the process of getting acquired. They just couldn't say anything about it. But it's definitely one of the biggest AI news of the year. And you can listen to what it's like or what's going through Jon's mind back then, as well as now today, six months later.
Welcome back to the pod.

**Jason Corso** (1:28)
Thank you so much. This is an interesting place to have the pod under the overpass of Interstate whatever it is.

**Swix** (1:33)
Yeah, Interstate whatever in the city of New Orleans. Yeah, it's really good to see you. Since you were last on the pod, Mosaic got acquired.
Yeah, thank you.

**Jason Corso** (1:42)
I think you really deserve all the credit for this.

**Swix** (1:44)
No, you guys are sitting on that news and we didn't know what was going to happen. But I did come away from your interview with a very, very high impression of, like, you guys are in a perfect place, perfect time, and it makes a lot of sense to join forces with Databricks.

**Jason Corso** (1:58)
Yeah, they're kind of, I mean, I will say we really didn't want to get acquired.

**Swix** (2:02)
We did not?

**Jason Corso** (2:04)
We didn't. I mean, we loved being independent, like, we loved doing our own thing.
But this just made too much sense. Like, you know, they do data, we do LLMs, we both do enterprises, we're all a bunch of academics. Like, it was just kind of, we couldn't think of a better match. And so it just, we kind of came to the conclusion like, okay, I guess we can't not do this. Like, it's too perfect.

**Swix** (2:26)
Yeah, yeah. And you've done a bunch of other podcasts on the acquisition, so I don't need to retry. I'll send people that way. Just like, what's new in Mosaic World?

**Jason Corso** (2:34)
In Mosaic World, honestly, like, we're just cooking. I think we've been a little quiet lately. Yeah. Or at least we look quiet from the outside.
It is certainly not that we haven't been busy and it's certainly not that, you know, we're not doing cool stuff. Part of it is that, you know, getting acquired, there's a bit of administrivia involved. You know, we had to go through new employee orientation, get health insurance, you know, meet our amazing new colleagues.
Part of it is like, you know, the field has moved toward bigger stuff and we've moved toward bigger stuff. So, I think we'll have some exciting stuff to talk about soon, but my philosophy is always like speak through the work. Yeah. So, I don't want to hype, I don't want to like get people excited, you know. You'll see the work and you judge for yourself.

**Swix** (3:10)
Yeah. You talk about the industry moving towards bigger stuff. What trends are notable to you in the, let's say, second half of this year?

**Jason Corso** (3:16)
Everybody's figured out how to build LLMs. Like it's no longer a coveted skill of, you know, a handful of people, but now we've all become LLM builders. The field has kind of narrowed in aperture again and, you know, in yesteryear when we were all figuring out how to train ImageNet, you know, now we're all figuring out how to build really big, really powerful models.