Beating GPT-4 with Open Source LLMs — with Michael Royzen of Phind

**Alessio** (0:09)
Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO of Residence and Decibel Partners, and I'm joined by my co-host, Swix, founder of Small AI.

**Swyx** (0:19)
Hey, and today we have in the studio Michael Royzen from FIND.

**Alessio** (0:22)
Welcome.

**Michael Royzen** (0:23)
Thank you so much. It's great to be here.

**Swyx** (0:24)
Yeah, we are recording this in a surprisingly hot October in San Francisco, and sometimes the studio works, but...

**Alessio** (0:31)
The blue angels are flying by right now, so sorry about the noise.

**Swyx** (0:35)
I don't think they can hear it.
We have enough time. So, welcome. I've seen FIND blow up this year mostly, I think, since your launch in Feb and V2 and then your Hacker News post. We tend to like to introduce our guests, but then obviously you can fill in the blanks with the origin story. You actually were a high school entrepreneur. You started SmartLens, which is a computer vision startup, in 2017

**Michael Royzen** (0:58)
That's right. I remember when like TensorFlow came out and people started talking about... Obviously, at the time, after AlexNet, the deep learning revolution was already in flow. Good computer vision models were a thing. What really made me interested in deep learning was, I got invited to go to Apple's WWDC conference as a student scholar, because I was really into making iOS apps at the time. So I go there and I go to this talk where they add an API that let people run computer vision models on the device using far more efficient like GPU primitives. After seeing that, I was like, oh, this is cool. This is going to have a big explosion of different computer vision models running locally on the iPhone. And so I had this crazy idea where it was like, what if I could just make this model that could recognize just about anything and have it run on the device?
And that was the genesis for what eventually became SmartLens.
I took this dataset called ImageNet 22K. So most people want to think of ImageNet, think of ImageNet 1K. But the full ImageNet actually has, I think, 22,000 different categories. Yeah. So I took that, filtered it, pre-processed it, and then did a massive fine tune on Inception V3, which was, I think, the state of the art, deep convolutional computer vision model at the time.
And to my surprise, it actually worked, like, insanely well. I had no idea what would happen if I give a single model, I think it ended up being 17,000 categories, approximately, that I collapsed them into. It worked so well that it actually worked better than Google Lens, which released its V1 around the same time.
And on top of this, the model ran on the device, so it didn't need an internet connection. My big part of the issue with Google Lens at the time was that connections were slower, you know, 4G was around, but like it wasn't nearly as fast, and so there was a noticeable lag having to upload an image to a server and get it back. But just processing it locally, even on the iPhones of the day in like 2017, much faster. It was a cool little project. It got some traction. TechCrunch wrote about it, and there was kind of like one big, you know, spike in usage, and then over time it tapered off. But people still pay for it, which is wild.

**Swyx** (3:14)
Oh, it's like a monthly or annual subscription?

**Michael Royzen** (3:16)
Yeah, it's like a monthly subscription.

**Swyx** (3:17)
Even though you don't actually have any servers?

**Michael Royzen** (3:19)
Even though we don't have any servers.
That's right. I was in high school. I want to make a little bit of money. I was like, yeah.

**Swyx** (3:25)
That's awesome.

**Michael Royzen** (3:26)
Thank you.

**Swyx** (3:26)
I always wonder, like, what the modern equivalents kind of be my eyes.
And they actually disclosed in the GBT4 vision system card recently that the usage was surprisingly, like, not that frequent. Like, the extent to which, like, all three of us have a sense of sight, I would think that if I lost my sense of sight, I would use be my eyes all the time. The average usage of be my eyes per day is 1.5 times.

**Michael Royzen** (3:48)
Exactly. And I was thinking about this as well, where I was also looking into image captioning, where you give a model an image, and then it tells you what's in the image. But it turns out that what people want is the exact opposite. What people want is to give a description of an image, and then have the AI generate the image.