OpenRouter: from Seed to Stripe — with OpenRouter’s Alex Atallah & AMP’s Anjney Midha

Swyx (0:03)

Okay, we are here in Anja's house. It was just where all great startups in San Francisco start. Howdy.

And congrats on Cursor, Mistral. I don't, God knows what else, you got so much stuff going on.

Anjney Midha (0:17)

There's a lot going on. Well, OpenRouter is probably the most, has been the most, I would say, one I'm excited about recently.

Swyx (0:24)

Yeah. And we have Alex, first time on the pod, but you've been in the IE a few times. I appreciate every time you've shown up for the community. Congrats. I just like, what a journey. When I was looking back at your past posts, one of the earliest principles that I saw you write as a sort of product person is Pub-Sub as a product principle. And I wanted you to maybe explain how you think about what should exist in the world.

Alex Atallah (0:49)

Yeah. The Pub-Sub piece, which was for early 2023, I didn't think about it until we talked like 10 minutes ago, is about how there is like a way of thinking about products as an intersection between subscribing to data and publishing data.

And marketplaces are an easy, easy, easy example of this. You have suppliers that are publishing some kind of product to a SKU. And the SKU is kind of like a Pub-Sub topic that a consumer is subscribing to and just going to like consume whenever they want. And humans consume in a very like a discreet ad hoc way. It's not very scalable. All their attention is on the topic when they're buying the thing, and their attention is nowhere else when that happens.

Agents and consumers of inference don't act like that. They're consuming continuously and they're changing the SKUs that they consume from all the time. So, OpenRouter is sort of like a blend between a normal API experience and a Marketplace, where we create model slug, we have the auto-router, we have all kinds of product SKUs that you can subscribe to, and then you can continuously derive value and make decisions based on those consumers.

Swyx (2:11)

Yeah. This is something that was more consensus now, but not consensus when you guys started, which was that there is such a demand for swapping models and changing things out, and that people would not use the native SDKs. I guess for each of you, what was your sort of realization moment that this would be it? You've given a talk at EIE about Alpaca as one of your inspiring moments.

Alex Atallah (2:35)

Alpaca, I can rehash the Alpaca moment for a sec. The very beginning at the end of 2022, OpenAI was the only game in town. It was like OpenAI, Cohere, and then a smattering of early attempts at open weight models. Yeah. When Llama came out in January of 2023, it was like, wow, really exciting. This is really big. It outperforms GPT-3 on one or two benchmarks. But you can't chat with it. It wasn't actually an engaging model. But it seemed like someone just needed to fix a couple of things and do some RLHF on it to get it all the way there. And Alpaca was the first model that I saw that did that. It only took $600 to do. A team at Stanford generated a bunch of synthetic data, fine-tuned Llama, and made Alpaca, seven billion parameter model. Maybe it was 13 billion parameters. And it was so good. I was just on an airplane using it.

In many cases, you could not discern a ChatGPT versus an Alpaca result. And I figured if it was this easy to make a model.

One, we have a whole new way of monetizing data for the first time. You can just take really valuable data and turn it into a service in $600.

And that cost will probably go down over time.

Swyx (4:04)

When you say monetizing your data as what eventually will become the MCP endpoint or as a training data for a model.

Alex Atallah (4:11)

Yeah, training data for a model. An abstract way of saying like, hey, I have this data.

Swyx (4:15)

Converse it into a model.

Alex Atallah (4:16)

Like it makes sense for me in my product, but like I could repackage it in the form of a model and sell it. And so it's just a whole new business model for the economy.

It also, of course, provides like, you know, a way of following what Frontier Labs are doing, but in a way that like a single developer or a small team of developers can roll on their own. And so whenever you have an example of that, like a breakout app that's doing really well, and then some kind of framework for imitating it in your own flavor, you have an immediate ecosystem of like, like an immediate ecosystem like should arise, because there's just a huge gap between the like decisions that the single company is making, and all of the variations in those decisions that like a wider ecosystem can create themselves. And so then, you know, you need a marketplace to like discover all of those services and all of those products. There wasn't any place on the Internet that like was like a home base for LLMs in terms of seeing how much they were being used and seeing who was using them and why.