Listener Q&A: AI Investment Hype, Foundation Models, Regulation, Opportunity Areas, and More

**Sarah** (0:05)
Hey everyone, welcome to No Priors. Today we're going to switch things up a bit and just hang out and answer listener questions about tech and AI.

**Elad** (0:12)
The topics people want us to talk about include everything from the evolution of open-source models to the balkanization of AI.
Elon AI, which I think will be super interesting to cover, regulating AI and AI hype in the investing environment. Let's start with the march of progress for open-source models. I guess Sarah, what have you been paying attention to and what are some of the more interesting things that you view happening right now?

**Sarah** (0:32)
There's nothing out there today in open-source that is like GPT 4.3.5 or enthralic clod quality. There's one player out in front and that's open AI, but I think the landscape has changed a lot over the last couple of months. Like Facebook Lama is quite good.
Many startups are just using it despite its licensing issues assuming Mark won't come after them. Then you have a number of other releases that have happened. Tomorrow just released a pre-training dataset, which seems quite good, Stability just released Diffusion XL in the ImageGen space.
I think the larger dynamic is that there's been an increasing number of people and teams that now know how to train large models. The cost of a flop is only going to go down. There's a lot of investment in distilling models. A lot of researchers would claim that you and I know that it's going to be 5X cheaper to train the same size model the second time around. Once you've made your mistakes and know what you're doing. Then you have these other accelerants. You can use these models to annotate your datasets and increasingly do advanced self-supervision. If VCs are going to continue to fund foundation model efforts, including open source foundation model efforts, if I were a betting woman and I am, I bet there's a three-five level model in the open source ecosystem within a year.
I didn't personally believe that would be true a few months ago.

**Elad** (1:54)
I guess that puts it about two to three years behind when GPT 3.5 came out though. And so do you think that's going to be the ongoing trend, that there'll be a handful of companies that are ahead of open source by one or two generations?

**Sarah** (2:06)
Yeah, I think that's like the status quo. So if we just straight line project, I imagine that will continue to happen. And the real question is, can you stay in the lead if you are open AI and get paid for that? Or is that the objective of the organization?
Anyway, I think if you have a great leader and a lot of resources and a lot of really talented people, that's not something I want to bet against.

**Elad** (2:32)
Is there anything you think is coming in terms of other big shifts in the model world, either on the open source side or more generally?

**Sarah** (2:41)
Yeah, I mean, we should also talk about just like stuff that you're interested in investing in and generally paying attention to. But I think the big idea that's been very popular over the last few weeks are autonomous agents, right? And I don't think that that's like a, I want to hear what you think about this too. I don't think that's necessarily an architectural change, but for our listeners, the basic idea is to orchestrate LMS in this iterative loop towards some high level goal, where they're doing planning and if memory and prioritization, reflection. And so you're not necessarily changing the architecture of the LMS itself, but this orchestration allows you to do many new things, possibly. The classic example being like, make money on the Internet for me. And there's a good number of hackers trying to figure out how to make agents that, for example, analyze demand, find a supplier, set up a drop ship Shopify store, generate ads, then promote that store on social, right? The whole loop being like one call to an agent with this high level goal of make money on the Internet for me.
Do you think this stuff is interesting around autonomous agents?

**Elad** (3:47)
I think it's super interesting. And I think there's the old saying that the future is here, it's just not equally distributed. And I feel like that's one of those things that people in the AI community have been talking about for a while, and there have been very clear ways to do it. And then I think there's one or two people that went and implemented interesting things there in terms of auto GPT or other things. And then everyone's like, oh, my gosh, this can happen.