SaaStr 861: Our AI Agent Negotiated a Vendor Renewal, Became a CFO and a Better SDR .. But Does He Have Too Many Guardrails?

**Amelia** (0:00)
I was like, well, the agent is the decision maker. I'm not going to decide whether to renew this tool. Literally 10K is going to decide for you. And if it says no, I'm going to say no. Like, for this, he has the budget.

**Jason** (0:11)
Requirements to do a renewal. 10K made the, for real, is making the decision whether to renew it or not, right? For real.

**Amelia** (0:18)
And so he came up with this short list of things he wanted to see in the API, and that was one of the things I sent over. And then second, 10K said to push them on pricing, because their pricing is still seat-based. And he said, well, we only need one seat. So, you should push them to charge you for one API power user and dump the rest.

**Jason** (0:42)
Hey, everybody. Welcome back to The Agents, everybody's favorite show about the trials and tribulations of managing 21 AI agents, three humans and a dog. We are back with a lot to reflect on. We got through SaaStr AI Annual, 10,000 of the best building and sharing with agents. By far, I think the best event we've ever done. On the last episode, if you listened to Amelia in particular, I went through what we've learned on building our agents. We're going to talk about that next. And now we got to figure out what we're going to do the rest of the year with our agents. And so that's a little bit of what we're going to talk about. I want to do a lot and talk about our team of agents, 10K, 10K Prime, and more. But first, I want to get to something, Amelia, you might not know this, which is regressions.
And while we're off doing our little get together with 10,000 founders, we have an app I built a while ago called our VC Pitch Deck Analyzer. It's free, it's at saastr.ai. You can just click on AIVC and it will grade your VC Pitch Decks for you. It's really good. And in fact, we have done, at least as I type this, we've reviewed 4,614 VC Pitch Decks. It's great, it's honest, it tells you whether your numbers are good enough, whether you get funded, what VCs will think, it will give you the honest feedback that humans won't. And then I noticed kind of the last week or two when we were busy with doing our big AI annual event, it was giving a lot of Fs.
So it gives you quantitative scores of all these metrics, growth, team, market size, everything. And then at the end of the day, it just gives you a grade to make it simple. And basically a B plus or higher means you'll probably be able to raise venture funding. You know, a B to a B minus means you're not, it'll be tough. You might be able to raise funding, but you probably got to talk to 100 VCs and, you know, a C plus and below is tough. And it was just giving F after F after F after F. And I figured it was just Yahoo's uploading lame decks or something like that. And I didn't have time to investigate until just a couple of days ago. And then I realized it was broken.
It was partially broken.
And it's an interesting story why I didn't break it. Did you touch it? You don't even know where it is, right? You don't use it. But this was interesting just for my learning. So of the last 305 submissions, 88 just failed. Just failed. And of the 216 that worked, 53% got Fs. They just got Fs. And what had happened was this was just interesting for figuring out how to deal with agents and guardrails. Because what happens is we've done thousands of decks. Obviously, I haven't done 4,000 or something myself. But I've run a bunch manually. Companies I've worked with, you know, owner raising around, revenue raising around, gorgeous raising around others, and I run it through. And then there'll be mistakes. Like for a while, I couldn't translate euros to the US the right way. And then it sometimes got confused on different types of forecasting and weird types of language. And so I kept adding guardrails, right? If it does this, do this, if it does that. And I guess it had reached so many guardrails that it rejected everything. It reached a threshold of guardrails. And I asked Replet what it thought the number was. And I think it said, by the time we'd added the 14th guardrail, the 14th exception, everything was an exception. Everything got bounced. And so everything started to break and we had to rebuild it from scratch and get rid of almost all of those rules, like throw away all these rules. And so I think someone smarter than me could probably, you could probably figure this out better. But it's interesting that months and months of layering these rules into this mega prompt, it was too many, it broke the system and it rejected everything.