Applied Wizardry: Cutting AI coding token costs, with Dave Slutzkin

Yaniv Bernstein (0:00)

You are listening to an episode of Applied Wizardry, the practitioner podcast for people who are using agentic AI to build real software products and real businesses. Hosted by me, Yaniv Bernstein. For now, we will be sharing episodes in this feed, The Startup Podcast Feed, but it will not be forever. So if you like the content we're producing, you want to make sure you get it first, and you want to get it without this slightly annoying message, go to Applied Wizardry in your favorite listening app or on YouTube and subscribe right now.

Can't wait to see you over there.

Dave Slutzkin (0:37)

Wait, where's all this money going?

Yaniv Bernstein (0:39)

Well, that's what this episode is about, right? Like, how do we get the same intelligence for lower cost?

Dave Slutzkin (0:44)

We've known for a long time that it's really important to understand your tools and to understand the detail of what's happening when you're building software. The same turns out to be true for these amazing and magical and complex tools.

Yaniv Bernstein (0:57)

Picking one model that you like that's at the frontier and saying, this is my model, that just doesn't work anymore. If you're listening at Anthropic, please don't sue Dave.

Dave Slutzkin (1:05)

If there's only one thing that you intend to care about, it should be.

Yaniv Bernstein (1:11)

Pablo Picasso once said, when art critics get together, they talk about form and structure and meaning. When artists get together, they talk about where you can buy cheap turpentine. Well, welcome to Applied Wizardry. This is the podcast if you are a practitioner and you want to talk about the cheap turpentine of building with AI. This is a podcast that talks about all the things that actually matter for you every day, practitioner to practitioner. Today, we have got the ultimate cheap turpentine guest, Dave Slutzkin, is co-founder of Antiburn, a company that helps you understand and optimize your token spend so you get more bang for your buck. Cannot wait to get into it with him. It is full of practical tips that will help you in your AI building journey right after this break.

This episode of Applied Wizardry is brought to you by Dam Secure. Dam Secure is the code security platform for AI-first engineering teams. Get top-tier security PR reviews at a fraction of the cost of other models and get through your PRs 32% faster. Don't just take my word for it. Head over to damsecure.ai.wizardry.

That's damsecure.ai.wizardry for your first month completely free.

One of the amazing things about AI is it allows you to turn your idea into a real business faster than ever before. But you don't want to let yourself get dragged down by paperwork, bureaucracy, and government stuff like that. That's why you need today's sponsor, SparkLaunch. For just $499, their Delaware C-Corp Formation Package includes Delaware filing and your first year of registered agent service. Plus, keep your company documents and founder equity organized as you grow, so you can focus on building the business. You build the product, SparkLaunch helps you start the company. Visit sparklaunch.io/wizardry to get another 20% off their already great deals and get started.

Dave, welcome to Applied Wizardry.

Dave Slutzkin (3:17)

Thank you, Yaniv. Great to be here.

Yaniv Bernstein (3:19)

I thought we should start with the problem, the problem of Token Burn and why is everyone talking about it so much?

Dave Slutzkin (3:25)

Feels to me like 10 or 15 years ago with Cloud, we're bumping in this situation where individual engineers are making decisions which are hitting your P&L really directly on a daily basis. They're doing these things that are starting to affect your budget as a CTO, as a CIO, and you might be getting tapped on the shoulder. What I'm more increasingly seeing is CTOs getting tapped on the shoulder by CFOs saying, wait, where's all this money going? People are spending $10,000 US in tokens per developer per month. That feels like a lot to me. It's significantly a fraction of a salary. So yeah, I think that's why people are starting to care about Token Burn at an organizational level. From a personal level, you've written a lot of code over the years, as have I.

You like to do things the right way and no one quite knows what the right way is, but they can tell when it feels like they're going on tangents and spending tokens and spending time and spending effort doing things that are useless. So I feel like that basic engineering mindset comes in as well. Personally, I see that.

Yaniv Bernstein (4:23)

I think you've got that right. There's been this real whiplash from a few months ago, people were talking about token maxing and like you said, there was this rush to AI where companies were giving their engineers unlimited token budgets until, as you say, the CFO came and tapped on the shoulder and they're like, okay, now we need to optimize. Nobody wants to just take AI away, right? It's one of those crossing the Rubicon things. You don't want to say, well, you have to really massively ration your use of AI and hand write more of your code and only use AI when you need to. What we really want to be doing is getting the same amount of intelligence, the same amount of work done, but at a much lower cost. And the truth is there is a lot of room to optimize.