Cloud Intelligence at the speed of 5000 tok/s - with Ce Zhang and Vipul Ved Prakash of Together AI

**Alessio** (0:05)
Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, Partner and CTO and Residence at Decibel Partners, and I'm joined by my co-host, Swix, founder of Small AI.

**Swyx** (0:14)
Hey, and today, we're Together with Together. Welcome to the studio, guys.

**Vipul Ved Prakash** (0:20)
Thanks for having us.

**Swyx** (0:22)
I don't know how you typically give self-intros, but does anyone want to go first? How do we get our audience acquainted, especially to who's speaking, because it's unusual for us to do a four-person pod.

**Ce Zhang** (0:33)
Yeah. Hi, everyone.
I'm one of the co-founders of Together and the CTO, working with the team on the technical things.

**Vipul Ved Prakash** (0:39)
I'm Vipul Ved Prakash, co-founder and CEO of Together.

**Swyx** (0:43)
I always consider you guys as one of the sort of all-in-one companies. I always want to say labs, but I feel like you're not a lab.
What is the origin of Together? And then what is it today? I feel like you used to be Together.XYZ, and then now you're Together AI.

**Vipul Ved Prakash** (1:01)
I think fundamentally, Together is about open and independent AI systems. We think this is one of the most consequential technologies of our time. And when we started the company in June 2022, our focus was to build a platform for open source independent user-owned AI systems. One way to think about it is big labs, frontier model labs have built their own developer platforms for their models. We think of Together as a platform for everything else.
Whether these are open models, whether these are models being built by companies that are owned by them. And our XYZ roots, we have a fairly deep decentralization and open ethos that reflects in all our platform and strategy and business.
And we also, the way we structure our cloud is by combining data centers around the world instead of, we are today not located in hyperscalers. We have built a footprint of AI supercomputers in this sort of a disaggregated decentralized manner.

**Alessio** (2:15)
I know before Together you were Apple. So you go from like the most wall garden, private. We don't say anything company to, we want everything to be open and everybody to know somebody.
What maybe did you learn from like the Apple way of being super close and polished? And maybe what are you taking now to Together to make it open, but also a very nice developer experience?

**Vipul Ved Prakash** (2:37)
Yeah, I would say, you know, one sort of my, you know, background has been in open source for a long time. One of the first things I created was collaborative spam filter. You know, this is back in the day.

**Swyx** (2:49)
It's called Vipul's Razor.

**Vipul Ved Prakash** (2:50)
It's called Vipul's Razor.
And it became quite popular. And the first company I founded called CloudMark was built around, you know, taking open source and building both an open side of it and a commercial product around it. I think Apple is sort of very focused on providing this amazing experience to its customers with, you know, most of the technology sort of hidden behind the product.
And certainly the focus on fluidity and applying complex technology to make everyday things simple is something that Apple does really well. And, you know, that's been a sort of big part of how we think about our developer platforms. I think it informs it.
The other thing is that during my years at Apple, we, you know, worked a lot on deep learning. And one of the things that was sort of very viscerally accessible to me was how well these systems worked. We, you know, we built an open domain Q&A system. This was based on Facebook's LSTM paper in 2016 And it was remarkable because we had a parallel system based on sort of information retrieval techniques. It was just extremely complicated. It didn't work that well. And, you know, this thing we wrote in a week was just incredible performance. So I think some of those experiences, at least for me personally, sort of were creating this roadmap of how important and powerful this technology is. And, you know, when the Scaling Laws paper was published, I was very clear, like it was, in some ways, something very profound. We've never had algorithms that improve in capabilities with scale out. So this is almost a new era of computing. So that's been, I think, the influence of Apple. My ears at Apple really, for me, like, crystallized the value of what we are doing together.

**Alessio** (4:54)
And how did you decide to join forces? Because you did a post-doc with Chris Ré at Stanford. You know, we already had Tri Dao from Together, and we talked about Hazy. What was, like, the meeting of the mind of, hey, I come from, like, the more technical post-doc assistant professor background, and Vipul, you had a more product thing. What got you excited to, like, build this now?