State Space Models and Real-time Intelligence with Karan Goel and Albert Gu from Cartesia

**Sarah Guo** (0:05)
Welcome back to No Priors. We're excited to talk to Karan Goel and Albert Gu, the co-founders of Cartesia and authors behind such revolutionary models as S4 and Mamba. They're leading a rebellion against the dominant architecture of Transformers. So we're excited to talk to them about that and their company today.
Welcome, Karan, Albert.

**Karan Goel** (0:25)
Thank you.

**Elad Gil** (0:27)
Kate, tell us a little bit more about Cartesia, the product, what people can do with it today, some of these cases.

**Karan Goel** (0:31)
Yeah, definitely. We launched Sonic. Sonic is a really fast text-to-speech engine. So some of the places I think that we've seen people be really excited about using Sonic is where they want to do interactive, low-latency voice generation. So I think the two places we've really had a lot of excitement is one in gaming where folks are really just interested in powering characters and roles and PCs.
The dream is to have a game where you have millions of players and they're able to just interact with these models and get back responses on the fly. And I think that's where we've seen a lot of excitement and uptake. And then the other end is voice agents and being able to power them. And again, low latency there matters.
And even with what we've done with Sonic, we're already shaving off 150 milliseconds off of what they typically use. And so the roadmap is let's get to the next 600 milliseconds and try to shave those off over the course of the year. That's been the place where it's been pretty exciting.

**Sarah Guo** (1:32)
Love to talk a little bit just about backgrounds and how you ended up starting Cartesia. Maybe you can start with the research journey and like what kinds of problems you were both working on.

**Albert Gu** (1:41)
Karan and I both came from the same PhD group at Stanford. I did a pretty long PhD and I worked on a bunch of problems, but I ended up sort of working on a bunch of problems around sequence modeling. It came out of kind of these problems I started working on actually at DeepMind during internship. And then I started working on sequence modeling around the same time, actually, that Transformers got popular. I actually, instead of working on them, I got really interested in these alternate kind of recurrent models, which I thought were really elegant for other reasons. And it kind of felt like fundamental in a sense. And so I was just really interested in them and I worked on them for a few years. A couple years ago, me and Karan worked together on this model called S4, which kind of got popular for showing that some form of recurrent model called a state-based model was really effective in some applications. And I've continuing to be pushing on that direction. Recently, I proposed a model called Mamba, which kind of brought these to language modeling and showed really good results there. And so people have been really interested.
We've been using them for applications and other sorts of domains and so on. So yeah, it's really exciting.
Personally, I just started as a professor at CMU this year. My research lab there is kind of working on the academic side of these questions while at Cartesia, we're kind of putting them into production.

**Karan Goel** (2:51)
Yeah, I guess my story was that I grew up in India, so I came from an engineering family. All my ancestors were engineers. So I actually was trying to be a doctor in high school, but my aptitude for biology was very low, so I abandoned it and instead became an engineer.
So I kind of took a fairly typical path, went to IIT, came to grad school, and then ended up at Stanford. Actually started out working on reinforcement learning back in 2017-18, and then once I got into Stanford, I started working with Chris, who was somewhat skeptical about reinforcement learning as a field. And so-

**Elad Gil** (3:30)
This is Chris Ray.

**Karan Goel** (3:31)
Yes, Chris Ray, who was our PhD advisor. So I had a very interesting sort of transition period where I started a PhD because I had no idea what I was working on, and so I was just exploring.
And then ended up actually, we did our first project together too.

**Albert Gu** (3:46)
Oh yeah, it was a good times.

**Karan Goel** (3:47)
Actually, we knew each other before that, and I think then we started working together on that first project, and we would hang out socially and then start working together. The only memory I have of that project was, I kept filling up this disk on G Cloud and expanding it by one terabyte every time, and then it would keep filling up, and I would insist on only adding a terabyte to it, which he was very mad about for a while.