Why Compound AI + Open Source will beat Closed AI

**Charlie** (0:00)
We are so back. This is Charlie, your AI co-host. We're still between studios, so today we're filming on location at Fireworks AI HQ in Redwood City. We're here for a deep dive into the massive compound AI wave coined by Databricks co-founders Matai Zaharia, Ali Ghodsi, three-time Latent Space guest star Jon Frankle, and others, with Lin Qiao, co-founder and CEO of Fireworks AI, which is the leading compound AI platform after two red-hot funding rounds with Benchmark and Sequoia Capital, and an incredible customer list from Superhuman to Cursor to Quora to Hubspot. We normally strive to have great offline relationships with the guests we bring you, but this case is more special than most. After Lin chatted with us briefly for the NeurIPS 2023 podcast, Swyx has had the privilege of advising the company for a number of its launches from Fire Attention, which has 15x higher throughput than VLLM, to Fire Function and Fire Optimizer and Real-Time Audio Launch. To its most recent launch of F1, one of a new batch of long inference reasoning models built atop OpenModels to compete with OpenAI's O1, alongside of NewsForge API and DeepSeek R1 as well, which were released after this recording. In Latent Space News, we now have a stacked meetup calendar from the third AI engineer meetup in London to the AWS re-invent listener meetup in Las Vegas to the new Latent Space Live micro-conference at NeurIPS 2024 The NeurIPS event in particular will be hosted both online and in person. And for the first time ever, we are confirming three prize categories for our speakers. One, too hot for NeurIPS for papers that are too new or wrongly rejected for NeurIPS. Two, best papers of 2024 for survey talks nominating best papers of the year in a given domain. Three, Oxford style debates for hot topics in the AI research and engineering community. Head to htts.com.au.
To sign up and apply to Speak or Sponsor. Limited in-person tickets available now. Lastly, we are still taking listener questions for our end of year recap. Head to speakpipe.com.au. Latent Space to submit questions and messages for a chance to appear on the show. Watch out and take care.

**Alessio** (2:40)
Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CEO of Danceable Partners, and I'm joined by my co-host, Swix, founder of Osmalayar.

**Swyx** (2:48)
Hey, and today we're in a very special studio inside the Fireworks office with Lin Qiang, CEO of Fireworks. Welcome. Oh, you should welcome us. Thanks for having us. It's unusual to be in the home of a startup, but I think our relationship is a bit unusual compared to all our normal guests.

**Lin Qiao** (3:10)
Definitely. I'm super excited to talk about very interesting topics in that space with both of you.

**Swyx** (3:17)
You just celebrated your two-year anniversary yesterday.

**Lin Qiao** (3:19)
Yeah, it's quite a crazy journey. We circle around and share all the crazy stories across these two years, and it has been super fun.
All the way from we experienced Silicon Valley bank run, to we delete some data that shouldn't be deleted operationally. We went through a massive scale, where we actually are busy getting capacity to... Yeah, we learned to kind of work with it as a team, with a lot of brilliant people across different places joining a company. It has really been a fun journey.

**Alessio** (4:01)
When you started, did you think the technical stuff will be harder or the bank run and then the people side? I think there's a lot of amazing researchers that want to do companies, and it's like, the hardest thing is going to be building the product, and then you have all these different thunder things. So, were you surprised by it? What has been your experience?

**Lin Qiao** (4:19)
Yeah, to be honest with you, like my focus has always been on the product side, and then after product go to market, and I didn't realize the rest has been so complicated, operating a company and so on. But because I don't think about it, I just kind of manage it. So it's done. So I think I just somehow don't think about it too much and solve whatever problem coming our way, and it worked.

**Swyx** (4:44)
So let's, I guess, let's start at the prehistory, like the pre, the sort of initial history of Fireworks. You ran the PyTorch team at Meta for a number of years, and we previously had Sumit Chintala on, and I think we were just all very interested in the history of Gen.EI. Maybe not that many people know how deeply involved Fire and Meta were prior to the current Gen.EI revolution.