Mamba-palooza: 90 Days of Mamba-Inspired Research with Jason Meaux: Part 2

**Erik Torenberg** (0:01)
Hey everyone, Eric here. We're really excited about a new AI show from Turpentine called Autopilot, hosted by Will Summerlin.
This podcast explores the adoption and rollout of AI in the industries that drive the economy, and the dynamic tech founders bringing rapid scalable change to slow moving industries. From law, to hardware, to aviation, we'll interviews founders backed by Benchmark, Greylock, YC, and more to learn how they're automating at the frontiers and entrenched industries. Click on the link in the description to subscribe to Autopilot.

**Nathan Labenz** (0:31)
Hello, and welcome back to The Cognitive Revolution. This episode is part two of my Mamba Palooza with fellow AI scout, Jason Meaux. If you missed part one, you might want to start there. If you haven't heard my original Mamba episode from last December, I'd recommend starting with that one for important foundational context. In this episode, we'll be covering Mamba's application to computer vision, experiments in extending the effective context window, and a bit of biology as well.
Once again, special thanks to Jason for putting in a ton of work to make this happen, and an open call to all of you to suggest additional topics for us to explore on the show, and especially to volunteer to go on one of these adventures with me. You do not need to have a PhD in machine learning to do this work. You just need extreme curiosity for the subject and a relentless drive to understand what's going on.
As always, we appreciate it when listeners share the show online, a tweet is worth a lot, and a review on the major platforms is especially valuable. With that, here's part two with Mamba Scout Jason Meaux, beginning with a discussion of a friendly bet that we made about just how many Mamba papers we each expected to see. Enjoy.
Okay, cool. So we are back. First, let's talk about the bet. We got an interlude.

**Jason Meaux** (1:47)
The bet was between you and me, maybe it was a few weeks ago. And so we decided, okay, in this two-week period in February, how many Mamba papers will be published?
I think I'm pleased with the results.

**Nathan Labenz** (1:58)
Yeah, this is definitely an object lesson in exponentials. Be crazy. In the process of setting this over under, I was thinking, okay, where are we on this curve and how fast is it going to bend? I said five and a half. So you wisely took the over. It quickly became clear that the over was going to win.
And then we revisited.
Did I reset or did you reset the over under?

**Jason Meaux** (2:22)
Yeah, you reset it. I thought, wow, 14 That's going to be a tough get for the reset.
But we got fairly close, right? 13 papers in two weeks.

**Nathan Labenz** (2:33)
Yeah, so 13 was the final answer. My instinct is to say, what's the over under for the next 90 days? It seems crazy to say that it would be more than 100 that would come out over the next 90 days.
But then again, maybe not.

**Jason Meaux** (2:45)
I'll happily set the line and then I'll let you take the over under. So yeah, 90 days from now, I'll set the line at 55

**Nathan Labenz** (2:55)
And that's more, 55 new ones over the next 90 days?

**Jason Meaux** (2:58)
55 new ones, yeah.

**Nathan Labenz** (2:59)
I think I have to take the over, right? Because that would be less than a doubling relative to the first 90 days.
Is the second 90 days of Mamba going to have more or fewer papers than the first? I think I have to say more. You can check back on June 5th to resolve our wager. So, let's get into vision. This is the next big deep dive section for us. And this one I think is, it's interesting because as you noted, it is the majority of the papers. It's been striking to see this much work on the vision modality, particularly because there was no aspect of vision or image processing in the original paper. I don't know why that is, but what theory would be that images are not sequential in the same way that all the other things are sequential, right? And in language, it's token to token. In music, it's proceeding through time and you're predicting waveforms or whatever.
In DNA, that's one base pair at a time. But in an image, you have this kind of different sort of challenge where it's not like there's a single order to the pixels, right? They're in at least usually a two-dimensional configuration. We could have additional dimensions, whether that's going from image to video, adding a time dimension, or even going into 3D imaging. So all of those are represented in this Mamba literature so far.