Why AI Agent Costs Spike 10x Day to Day | Vinay Seshadri

Sophia (0:00)

What's next for Chargebee? What can people expect to see come from you guys over the next year?

Vinay Seshadri (0:05)

Yeah, Chargebee is building out agents. We are building out agents that help the finance office. We have receivables agents coming on online, and we have a growth agent coming on board for growth managers. And we're really leveraging all the unique data that we have to come up with better insights and recommendations on how you can grow your revenue and your margins.

Sophia (0:29)

Hello, everyone, and welcome back to MTS. Today, I am joined by Vinay Seshadri, who is the Senior Director of Product Management for Monetization and PLG Solutions at Chargebee. What a great title. And Chargebee is building billing and monetization infrastructure for software and AI companies. But today, wanted to talk to you because Chargebee just announced they launched AUDR, the Agent Usage Detail Record, which is an open standard designed to track who initiated an AI agent run, and what the run consumed across models, tools, and infrastructure. And Vinay, you actually led that work. So welcome to MTS.

Vinay Seshadri (1:06)

Thank you, Sophia. It's great to be here.

And yes, I mean, that's a great intro. You really set me up.

Sophia (1:14)

I really got to do the big setup. But yeah, we'd love to ask you about what was launched. So what exactly is this Agent Usage Detail Record? So how does it work? What is it? Why is it important?

Vinay Seshadri (1:25)

Absolutely, right. So we at Chargebee, we work with customers building for the AI era, like Code Rabbit, Gorgias, DeepL, Lambda Labs. And increasingly, one of the questions that AI companies are trying to answer, and it's surprisingly hard to answer, is how much did this customer's agent runs cost me this month? And what's the margin I'm making on the customer, right? So it's really difficult because, not because the data is hidden somewhere, it's just distributed in different places, and it needs to be aggregated and joined. And thinking about this problem, we looked back at it, and then the telecom industry has actually had a similar problem 40 years ago. Like when you made a call, the call traversed two different carriers, three different switches, and each switch kind of knew how much the call cost at that leak, but no one carrier could answer, how much did the call cost across the board, right? So they didn't solve that by all using the same billing system, for example. They solved it by agreeing on a data standard. They agreed on the call detailed record, which captures who the caller was and how much was consumed by the caller for that particular call. And then each carrier emitted it, and it became possible for us as users to have extreme flexibility in terms of roaming, interoperability of different services across different carriers around the world.

And in the end, receiving a very transparent bill that has line items for every single call that was made, when it was started, when it was ended, and what the rate was.

So, inspired by the call detail record by the telecom industry, we're launching the agent usage detail record, which is an open data standard to try and capture the same thing. How much does an agent run cost across the application, which knows about the user, the harness, which knows about all the different tool calls, and the routers that know all about the models that were called and the token consumption that happened.

Sophia (3:32)

Yeah. Now, this is pretty interesting because so far, SaaS has been fairly easy-ish to economically track. You say, okay, we have these many people and seats, and we pay this much a month, X, Y, and Z. Now, I mean, I experienced this as well on our own team. We all have a cursor subscription, but it changes.

This person uses $100 of credit, someone else uses $1,000 of credits. It really changes by workflow, by the actual use case. So, can you just describe to me, why is this so hard? Why is it so hard to actually determine the economical cost of software or of these agent runs?

Vinay Seshadri (4:12)

Yeah, I mean, the way I like to think about it is every AI company can now chart a revenue chart, but then cannot chart the margin chart for their business, right?

And that's because your usage itself, your usage patterns are going to vary month to month pretty heavily as a user. And it can vary day to day heavily, the models that you use can vary from expensive models to cheaper models, the tasks can be more compute intensive or tool call intensive. So because each different agent run can require different resources, the cost can be 10x for the same user from day one to day two, right? The cost of a single agent run can be 10x comparatively. So it's not that software hasn't had variable costs in the past. Snowflake, Twilio, all of these companies deal with variable costs per incremental customer for a long time now. It's just the magnitude of those variable costs have increased and then the predictability has dropped for those variable costs as well. So it's really important to instrument and measure the costs at a granular level today in order to be able to build sustainable business models and margin healthy business models in software today.