Future of LLM Markets, Consolidation, and Small Models with Sarah and Elad

**Sarah** (0:05)
Hi, listeners, welcome to No Priors. Today, Elad and I are just hanging out. We're going to talk about LLM consolidation, what's going on in chips. I think an interesting dynamic around what type of risk you should take as an AI company in pushing the envelope and some big transactions. So let's get into it. First topic, Elad, are we done here? Is it too late? Is the LLM market consolidated?

**Elad** (0:27)
It's such an interesting question, right? Basically, what we're seeing is a number of model companies are either having their teams join larger enterprises, so that may be parts of Inflection or parts of Character or parts of other companies, parts of ADAPT, AWS, and so there's sort of one dynamic going on there with the model side. And many of those companies are continuing to exist, right? Like some of the products are still running and being used in different ways. At the same time, there's enormous capital modes emerging to get to the biggest scale for foundation models. And so if you look at it, these companies are now raising billions or tens of billions of dollars, often either from hyperscalers, so the Amazons and Microsofts of the world, or from solvers, right? Because those are the only people who can actually give you billions and billions of dollars. The venture capital industry is just too small to actually be able to support the next round for these companies. So everybody's kind of partnering up. And so it's a really interesting question to ask, well, for all the other players in the market, where are they going to get these ever rising amounts of capital? And who do they partner with? Does Apple end up with a partner? Does Samsung end up with a partner?
Does XYZ, other company, end up with a partner? And so you can kind of map all the potential partners to all the model companies and just ask, how does all that fall out? And then in parallel, the big hyperscalers have an incentive to fund these companies simply because it, in some cases, also translates over to more cloud utilization as an industry in general. The incentives start to flip between BCs, clouds, other strategics and sovereigns in terms of what they want to do. It does seem like it's increasingly hard to think that most companies will end up being competitive outside of a fundamental breakthrough in the model architecture or cost of actually training and then running inference on the model or doing the post-training side. So I think it's a really interesting open question, but it does feel like we're moving into a stage of more and more consolidation. I don't know, what do you think about it?

**Sarah** (2:17)
Most of that makes sense to me. I would argue that the market has become more competitive, not less over the last year and a half. Maybe it's competitive between a set of players that have, like as you described it, a capital mode that there's some breakaway scale. But the dynamic now, at least from the consumption side, is there's continual and aggressive performance increases and competition on the benchmark and price decreases and also real open source players. And so you can have consolidation and people not necessarily making money yet.

**Elad** (2:51)
I think you actually raise a really interesting point, which shouldn't be under discussed, which is that API costs have dropped something like 200x in the last 18 to 24 months or something along that order of magnitude. David on my team actually pulled together a chart of all the pricing for all the various models and what that looks like over time. And it's dramatic in terms of how cheap the dollars per million tokens has gotten. And so I think related to that, if you have a 200x decrease in the cost of running these things, or inferencing these things, and part of that is distillation, part of that is like, what are you actually using in terms of the generation of GPU, etc, etc. The actual margin available and the revenue available is increasing from the perspective of usage, but it's becoming harder and harder to just go out and compete with the model, at least as an EPI business, right? And so that kind of pushes you into specialization and to other areas of doing either bespoke specialized models or specific types of post-training or vertical applications or things like that.

**Sarah** (3:56)
I think the other way that you could look at the consolidation is just like, what is the argument for capital at that scale from a business perspective? And like the mostly unsaid thing is like really like it's still AGI is the business, right? There will be emergent models, emergent behaviors and capabilities in the model where it will figure out how to make money for us, or it will be obvious, like how valuable it is. But I think in the more, you know, immediate and I'm not even saying that's wrong, but I'd say in the more immediate and like two to three year horizon, you know, you have consumer as a business, either apps like advertising or subscriptions, and nobody's gone the advertising route in anger yet or enterprise as a business. Both of these are real today.