Ep. 020 - Anthropic vs OpenAI Usage, Margins, Meta Compute, Future of MSL (Tokenomics) | Crystual Huang, Max Kan, Joey Brookhart, Jordan Nanos

**Jordan Nanos** (0:05)
Hello everyone, welcome back to SemiAnalysis Weekly. We're here with Episode 20 with the Tokenomics team. Things are moving fast. We are recording this on Wednesday, July 15th. I assume by the time this episode comes out, there's going to be a lot more releases of models, disclosures from these companies about their profitability, revenue, how many active users they've got.
Regardless, we're going to record a point in time right now, talk about token budgeting, Meta's Compute Strategy, MSL Futures, the release of Fable, Soul and Anthropic's Profit Margin. So excited to dig in with me today. We've got Max. How are you doing, buddy?

**Max Kan** (0:44)
Great. Thanks for having me.

**Jordan Nanos** (0:46)
Welcome. We've got Crystal. How's it going, Crystal?

**Crystal Huang** (0:49)
Hey, Jordan.

**Jordan Nanos** (0:51)
We've got Joey.

**Joey Brookhart** (0:53)
Hey, Jordan.

**Jordan Nanos** (0:53)
Cool. We're going to start with token budgeting. Crystal, over to you.
We've got lots of conversations that we've been having with enterprises over time and just asking them, like, what's going on. At Semi Analysis, we're still token maxing, but others are moving into the time of austerity. Can you take me through a little bit about what you've found with this article?

**Crystal Huang** (1:13)
So there's a lot of enterprises right now that are cracking down on their budgeting because they think their employees are spending too much. And a lot of people are getting scared because they're like, what if I don't have enough tokens to do my work? I don't think that's necessarily true and it applies to most people, right? Because a lot of people aren't even getting close to that limit. There's probably a few handful power users at each of these companies, which will really be affected.
But other than that, I think it hasn't been as big of an impact, I think, on most people's workflows as it has been portrayed on social media and stuff. There's been a lot of different strategies that companies have been imposing, whether it's like on a per person basis or like a monthly budget for the entire company from what we've seen. And I think the per company basis obviously probably makes a lot more sense than like on a per person basis just given that some users will generate more like use from it than others.

**Jordan Nanos** (2:05)
And how do you see people actually enforcing this? Like clearly you can burn through a budget faster if you're going with the super ultra premium max thinking model. But in some cases, when you're actually enforcing a budget, you take different approaches, right?

**Crystal Huang** (2:18)
Yeah, yeah. So I've seen some people actually, so I was talking to some people and they are like a smaller company and they have a much smaller budget. So the way that they do it is they try to optimize how they're using the like more expensive tokens for like Anthropic or OpenAI. So they use like a cheaper model to like process a lot of it and like condense it down to like a smaller prompt. And then they use the more expensive models or they use something that their company's not counting. So there's a lot of like discrepancies in terms of what models actually count. Some people or like some enterprises are counting their own models into the budget, but some enterprises aren't.
So there's like, if they aren't counting it, a lot of people will tend to like really push those to the limit, use those a lot and then use like the ones that actually cost money.

**Jordan Nanos** (3:05)
And you're seeing people, maybe Max I can bring you in on this one.
You've been seeing a lot more people burning tokens for coding rather than other use cases, right? That seems to be the market that's growing the fastest.

**Max Kan** (3:18)
Yeah, I think coding slash like software engineering in general is just by far the most token hunger use case. And I actually think this is why a lot of the token budgeting discourse is pretty, like I think a lot of the budgets themselves are pretty uninformed. Like I hear people saying that they want to make sure their sales guys aren't using Opus or Fable to write email. So they should definitely be using Sonnet instead. And it's like, dude, like generating your email is basically free from a token perspective. Like it does not matter if you use Opus or Sonnet to write this email. Like I have no idea why this is the policy you're enforcing to try to like reduce your token spend.
I also heard lots of stories, like certain companies where only the engineering team is allowed to use like Cloud Code or Codex. And then, you know, everyone else only gets like co-work or something like that. I think that's pretty short-sighted. And it's honestly, you shouldn't have a cast system where your engineers are at the top and everyone else is just forced to use like demo AIs, you should really be giving everyone the opportunity to try these tools and figure out how you can become more productive with them.