Ep20. AI Scaling Laws, DOGE, FSD 13, Trump Markets | BG2 w/ Bill Gurley & Brad Gerstner

**Brad Gerstner** (0:00)
And this was Friedman opining in a very quick rapid-fire interview of the 14 government agencies that existed at that moment in time, which he would abolish and which he would keep. And, you know, we'll roll it and put a clip in here. Keep them or abolish them? Department of Agriculture.

**Bill Gurley** (0:20)
Abolish. Gone. Department of Commerce. Abolish. Gone.

**Brad Gerstner** (0:25)
Department of Defense.

**Bill Gurley** (0:26)
Keep. Keep it. Department of Education. Abolish. Gone. Energy?

**Brad Gerstner** (0:32)
Abolish. Except that energy ties in with the military. Well, then we shove it under defense.

**Bill Gurley** (0:38)
The little bit that handles the nuclear.

**Brad Gerstner** (0:40)
Right. That ought to go under defense.

**Bill Gurley** (0:41)
Plutonium and so forth goes under defense, but we abolish the rest of it.

**Brad Gerstner** (0:44)
But, you know, they asked him, Department of Agriculture. Abolish. Commerce. Abolish. Education. Abolish. Hey Bill, we were in my office last night, my boys put on the cowboy hat, the cowboy hats we made down at your anniversary.

**Bill Gurley** (1:10)
Yeah, it looks great on you.

**Brad Gerstner** (1:13)
It's great.

**Bill Gurley** (1:14)
You should wear it for the whole episode.

**Brad Gerstner** (1:17)
It's good to be here.
It's kind of a surprise. This morning, I got on the plane to fly up to Seattle. I was gonna do a pod with Satya, as you know, and then we were gonna do our reaction pod, and a typhoon or something hit Seattle last night. Literally, like halfway up there, I got word that Microsoft has no power, nobody has any power, trees are downed. So I did a U-turn and wish them well. I'm gonna go back up there on Monday and do a pod with Satya. But given that, I couldn't wait to get together with you and kick around all the things that we've been sharing back and forth over the last couple of weeks.

**Bill Gurley** (1:52)
Lots of stuff happened.

**Brad Gerstner** (1:54)
No doubt about it. One of the things certainly popular in our threads has been AI scaling laws. Are we beginning to see models top out, particularly on pre-training? In fact, there was this Bloomberg headline, Open AI, Google and Anthropix struggled to build more advanced AI. It then goes on to say that Orion or ChatGPT-5, Gemini, and 3.5 Opus are all falling short of internal pre-training targets. Then they quoted Dario has saying, scaling laws are not actually laws of the universe, but they are simply empirical regularities. He said, I'm going to be in favor of them continuing, but I'm not certain of that. Yeah. So what's been on your mind about whether or not we are in fact seeing continued gains from pre-training?

**Bill Gurley** (2:50)
Well, and I would add a few other things.
Ilya was also quoted as questioning whether the scaling laws were continued. And for whatever reason, Mark and Ben over at dangerous and horrible it's also made the same statement. So in a very short window, we got this point of view from quite a few number of people. I would add that in Dario's five-hour interview with Lex, which I only grabbed pieces of from the transcript, but he was very positive on scaling laws in that interview, which just recently dropped.
So there may be a disagreement between some people, but the breadth of the feedback suggests something's up. I think something's worth paying attention to. And I would add that there were people that raised this question early on. And I think it's specific to LLMs. I don't think you should say AI scaling laws. I think that there was always a question about whether LLMs would run out of steam. And that was tied to three different things, which you and I talked about on July 11th. And I'd even raise this question a year ago. One is, will the parameter count run out? Historically, mathematical algorithms that do some type of fit. And this is a very sophisticated form of that. But when you take the variables up to a certain level, it stops adding value. You just get too close to the fit. There's a question about how big the context window will be. And that came up on the NVIDIA Call of Day. I think we can talk about that. And then data was brought up. And a lot of people, I know this was a big point Melanie Mitchell had raised, like, are we just going to run out of data? And there was pushback that there's synthetic data, but there's been papers published that show synthetic data creates a lot of chaos. And so I do think there were people that were thinking this might happen. And there was also a belief, and I share this belief, you don't have to, so we can disagree on this, that there was a argument being made by OpenAI and Anthropic that, oh, you're just going to see the next number drop, and the next number is going to be way better than the last number, and that's going to happen routinely. And they both said, we spent whatever, a billion on this model, and we're going to spend 10, and then we're going to spend 100 And that implies, I would say, at least linear scaling or maybe above. And they all said that. And so if the comment that you read earlier is true and that they're not getting the benefit, there are implications. It doesn't mean AI is in trouble or AI is done. I mean, there's tons of positive AI news out there, but it may mean we're shifting directions, and it's worth, I think, talking about, well, if this is true, what are the implications?