True Enterprise AI Agents Are Still 5 Years Away | Aghi Marietti

Aghi Marietti (0:00)

In our enterprise cohort throughout our AI gateway, there is several hundred of enterprise customer. On one trend, yes, there is open-weight adoption in the last nine months, but it's still actually minimal if you take the largest enterprise cohort. It's like under 5%.

SPEAKER_2 (0:17)

Hello, everyone, and welcome back to MTS. Today, we are joined again by Aghi Marietti, the co-founder and CEO of Kong, also known as the API Godfather, always a really fascinating conversation when you join us on MTS. Excited to get into a bunch of different things, Anthropics IPO, price of tokens, agents in the workplace, all of these great topics. But, Aghi, welcome to MTS.

Aghi Marietti (0:43)

Thanks for having me, Suviyah.

SPEAKER_2 (0:44)

Amazing. Well, thank you for joining us. As always, I think there's some really interesting stuff. I mean, you previously shared on MTS that customers are increasing how they're thinking about and using open weight models, and at the same time, you saw that Anthropic, of course, they're going public pretty soon, and they cut their token prices on its new models.

Curious to get your perspective on what is going on. What's the secret story here? Should Anthropic be worried about customers increasing their use of open weight models? What's happening?

Aghi Marietti (1:19)

I'll tell you, one of the data we have, working with enterprise customers is, so there's kind of two stories. One is like, oh, all the world is going to open weight model, and the cost per token is going down to one cent, and there is another one where the close weight model is always frontier intelligence and also growing revenue.

In our enterprise cohort through our AI gateway, there is several hundred enterprise customer. On one trend, yes, there is open weight adoption in the last nine months, but it's still actually minimal if you take the largest enterprise cohort, it's like under 5 percent. But then you have data of like SMB routers or long tail, and small company providers like OpenRouter, Versel, CloudFrame, where you have a spike of open weight model even 20-30 percent. And I think that makes sense, right? The long tail will always go for the cheapest intelligence to get stuff done. But in large, large, large enterprise, most of the spend is still on the usual suspect, whether it's Anthropic or OpenAI, so it's still there. So we see the same. And I think that will stay in large enterprise that just more like secure, adverse, and there are more kind of compliance and governance, and so there is kind of true stories of routing and cost per tokens. I think Anthropic cutting is good to have cheaper, will create more demand.

That's probably your strategy. But I would say enterprise are kind of the other side of the spectrum, and they're still going for the usual suspects.

SPEAKER_2 (2:54)

You say there's two stories. What do you mean by that?

Aghi Marietti (2:58)

It's like you have the long tail story, like you're going to access all these routers or showing data where open wait are growing like crazy in usage or token consumption, which is true.

But that's one side of the world. And also is the more like noise versus senior, right? Like the millions of developers, you get a lot of like out facing wars out there. Enterprise are much more quiet by designs and they use a huge amount of tokens. But their story is not, hey, we're going fully open wait just yet. I think they actually are going there. But what we see only less than 5% of all the token traffic with proxy where like about approaching one quadrillion run rate of tokens is actually open wait in the largest enterprise cohort. In the mid-size enterprise, yes, is much more open wait usage.

SPEAKER_2 (3:49)

So do cheaper tokens from Anthropic mean cheaper AI bills for everyone?

Aghi Marietti (3:56)

Yeah, I think so. I think so. But in this way, the intelligence is already here to do most of the stuff enterprise have to do. But we can talk about why they're not agentic this year or even next year. But you have a ladder where you use the smartest model to really do mission critical tasks. Let's say it's a mission critical pull request that needs to go into some system code where you need to have five, nine SLA. Those are get use and you want to use the most expensive intelligence to do the most expensive task. But then you have cheaper type of inference, cheaper type of models, and those ones are more your everyday task. So I think we're going to this bottom where maybe 80 percent of the intelligence consumed through enterprise is going to be on the cheaper side, and then you keep the 20 percent for the most advanced or frontier. They can be reviewing the workload of the cheaper ones. I think that's the world we're going.