The Shift to Agent-Driven API Consumption | Speakeasy’s Sagar Batchu

**Andrew Zigler** (0:06)
So, welcome to Dev Interrupted. I'm your host, Andrew Zigler.

**Ben Lloyd Pearson** (0:10)
And I'm your host, Ben Lloyd Pearson.

**Andrew Zigler** (0:13)
This week, we're talking about how AI can talk to each other now, how it can use tools, and ultimately, the cost of politeness.

**Ben Lloyd Pearson** (0:22)
Well, as someone who considers myself a polite person, I want to start there.

**Andrew Zigler** (0:26)
Okay, so this was a story this week where people really broke down the cost of saying things like please and thank you to ChatGPT. In this case, it was really going deeper at the cost of, saying those words requires the LLM to process them, just like any other part of your query, which ultimately uses more electricity, affects more of the environment, and people were even quantifying it in terms of spend, saying that being polite to ChatGPT costs the platform tens of millions of dollars. What do you think of this kind of polite madness?

**Ben Lloyd Pearson** (1:02)
Well, first of all, what if being polite gets you better answers and you don't have to do a second prompt? Well, then suddenly it becomes an energy saver, does it not?

**Andrew Zigler** (1:11)
I completely agree with that. Really, the breakaway here is people were arguing that saying please and thank you is a powerful way of expressing your meaning as part of your request, but it also helps make the LLM understand the importance of what you're doing compared to other parts of your request. Ultimately, those words have function, right? It's like we say them to each other and we think that they might be empty. Maybe we don't need to say them to a machine, but semantically, they're playing a role in a lot of people's requests. But you know what, Ben? The one where maybe I agree with, maybe we could get rid of it, is ending the chat session with just a thank you. I'm not sure what that does.

**Ben Lloyd Pearson** (1:49)
Do you do that? A whole extra prompt. So I used to be polite with my AI models, but I've come to think that they probably appreciate brevity over politeness, you know. They have a natural tendency to be pretty polite, so I think they just kind of want to get to the point.

**Andrew Zigler** (2:05)
Yeah, it's something I do commonly in scenarios where I get results that I do or don't like from an LLM, is I ask it to basically export the instructions or the tone or the concept of this conversation so I could share it with another LLM. Those kinds of kind of like plug and play prompt things are always very brief because the LLM knows that's talking to another LLM, it doesn't really need to be all that polite.

**Ben Lloyd Pearson** (2:28)
Yeah, and I mean, speaking of which, we've covered on here how AI chooses more efficient languages when it's speaking to other AI bots, you know, the human language is actually kind of inefficient when it comes to conveying meaning, but, you know, ultimately, I think it's kind of a waste of thought to think about the energy usage of stuff like this, you know. For one, like these things always become more efficient over time, like we're already seeing massive efficiency gains with AI. That's only going to continue and then we're going to get to a point where the models that people use day to day, like maybe there's going to be some extreme ones that take a ton of energy to run, but the ones that we use day to day are probably going to be very efficient and pretty cheap to run. But, you know, if you want to be polite, I think it's a good habit to have. So just do it. It's probably more efficient to speak to AI in emojis anyway. So if you're not speaking in emojis, then you're already not as efficient as you can be. But, you know, make a habit of being polite. Just don't expect that you're going to like save the climate from doing so.

**Andrew Zigler** (3:28)
Completely.

**Ben Lloyd Pearson** (3:29)
Yeah. So let's talk about the AI talking to each other. What's our story on that one?

**Andrew Zigler** (3:34)
You know, we were talking just a moment ago about how we communicate with AI and how we can be polite or not. Well, new on the scene from Google, if you're listening to this at Google Next, they announced the agent to agent protocol, which is a new protocol that allows an AI agent and AI tool to call out to another AI in a structured, repeatable way that allows then developers to build these interactions at scale. It's similar to other things that have come on to the ecosystem lately that we're going to talk about a little bit more in just a moment. But agent to agent protocol or a to a protocol, if you've seen this come across your feed or in news, really just to break this down for you, it's kind of like the idea of your LLM in a local tool or in your client being able to call out almost like an AI call center where there's another AI agent somewhere that can provide more in-depth information related to the tool or the process that you're doing. For example, if you have a local tool that has an AI built into it and it needs to access maybe cloud data information on a third-party service or otherwise interact with another AI workflow somewhere, it could use a to a to have a structured and repeatable conversation that both sides can understand. So this opens really a whole new paradigm for how software developers should be thinking about building their products for LLM consumption. It's really showing that developers are not the only people that are going to be using your tools. There's more dimensions in which AI will be acting in their behalf.