EP. 242 - LLM Gateway: One API Key for Every LLM

Ismail Ghallou (0:00)

There is this third-party company called Compute SDK. They do this weekly benchmark. We actually ranked first on August 7th. And if you go check who are we competing with, it's like multi-million billion-dollar competitors. We're only two guys behind the LLM Gateway.

Narb (0:19)

Welcome to the DevNTell Podcast, where founders, hackers, and anyone in between showcase their products built on the bleeding edge of tech.

Hello, hello. Welcome back to the show, boys. I'm Xtack the Hattie the Con.

Luca Steeb (0:32)

Yeah, thanks for having us.

Narb (0:33)

Hello.

Of course, of course. My pleasure. I'm really keen on folks kind of getting reintroduced or introduced to you guys because your guys' story is kind of core to why I started this podcast actually in the first place. Just kind of seeing folks start with a passion project and kind of turning it into a success like you guys have. But I guess before we get into that, would you guys just like to give a introduction about yourselves?

Luca Steeb (1:02)

Yeah, so I'm Luca.

My whole life I was a software developer working on different projects for different companies. I also was always interested in building my own stuff. And I've known Ismail for a long time, I think almost a decade now. And we've always collaborated on things, met a few times in real life. And around one and a half years ago, we started LLM Gateway because we were working on AI stuff, on automations, on, you know, I'm presently more into DevOps.

And we realized that it's like AI models are very, even back then already, very, like, you know, a bit into different platforms and it's very fragmented. And every API has different integration, has different integrations, and you need to sign up on different things, there's different billing. And then, like, some of the platforms, they don't report any, like, usage insights and you have to wait for a day. And then this is why we kind of decided to, there must be a better way to do this and then sell us LLM Gateway.

Narb (2:12)

Ismail, do you want to introduce yourself?

Ismail Ghallou (2:15)

Yeah, sure. I'm Ismail, also known as Smokosh on X and everywhere else. As Luca said, we've known each other for almost a decade now. I started as a designer, then turned into a developer, but nowadays I've enabled myself as just a builder.

And I'm also working full time in my job at the Token Factory. I call it the Token Factory because most developers know I just make home chain and consuming tokens.

Yeah, I had the same idea to provide one API to connect all these providers and models.

And when Luca reached out on Telegram, he was like, are you interested to join and build this? My answer was yes. And we immediately started building. There was like no planning, no meetings, no product strategy or market strategy. We just went on and started building. I was using back in the time, I think Cursor, I was using Cursor and V0.

We started by coding, but we still like it. We were manually checking the code base, doing smaller PRs. And because back then, AI was not really good for DC code. But over time when it was 4.5, I think, when we started just trusting the code it produced. But even though we were still verifying and reviewing some code. But lately, especially when Fable, Pipe and Natron Astra came out, we don't read the code anymore. But we're somehow doing this agentic coding. We have another AI that reviews the code, every PR, we're using CodeRavit. It's free if you have an open source product. But in our workflow, we generate the code, we test it. We have AI test it, it records the recording. We have a bunch of skills. Like we pretty much made code really good for AI to produce less slope in the future.

Narb (4:22)

Yeah, yeah, amazing, Fables. And yeah, indeed, it's quite interesting to see or reflect on how much AI has grown in terms of capability. It's like, yeah, I remember, yeah, back in the day, back in the day, here you go. Yeah, I was wrong with Cursor too. I was kind of hesitant on like, oh yeah, like, oh, okay, this is cool, can get rid of some boilerplate stuff for me. And then, yeah, like you said, like when Opus 4.5 came out in like December, I was like, oh, this is not pretty good. So, yeah, yeah, it's, I think, I think it's just gonna, gonna keep going unless external forces act against it.

That's another podcast. But I guess, coming back to LLM Gateway, I guess for folks, I guess you guys kind of summarized in your intros, but I guess concretely for folks who might not have caught the previous episode, you guys just want to give the overview of what LLM Gateway is.