Big tech earnings and the current AI debates, with Sarah Guo and Elad Gil

**Elad Gil** (0:05)
Today on No Priors, we're having a special episode of Sarah and me just talking. Hello, Sarah, how are you?

**Sarah Guo** (0:14)
Hey, Elad, what's going on? I see you a lot.

**Elad Gil** (0:16)
Not much. Good to see you.

**Sarah Guo** (0:17)
Let's talk about models. What's going on in the model world?

**Elad Gil** (0:21)
I guess there's a lot of hand models that are emerging. So I was thinking of maybe trying to do that eventually.

**Sarah Guo** (0:27)
It's almost as good of a business as investing.

**Elad Gil** (0:30)
I know, right? So there's been a lot that's happened in the model world recently. Obviously, Google launched Gemini, which I think had a few interesting characteristics both in terms of performance but also the huge context window, right? It was million token context window. Companies like Magic, I think in the past, have actually put out a 5 million token context window model and things like that, but it's really exciting to see that. I think for certain application areas, like biology, longer context windows actually seem to be quite important.
For example, if you're doing a protein folding model and you have a short context window, you're often actually not encapsulating much of the protein. The average protein is, I think, something like 300 amino acids long, at least in the human genome, but there are things that are dramatically larger than that. You just can't capture it in some of the context windows being used for biological models. I do think this is going to be one of those areas that will end up being more important than people think, at least in the short run.
But Gemini 1.5 seems to have some really interesting performance characteristics.
There's obviously Sora from OpenAI, which was the video model that is just beautiful to watch. There's other model companies like Pica and others that I think are doing exciting things as well. And then Mistral or Le Mis launched Le Chat, which is really the name of the product.

**Sarah Guo** (1:43)
Le Big Model.

**Elad Gil** (1:44)
Le Big Model, Le Big Mac.

**Sarah Guo** (1:45)
I believe they call it Mistral Large, yes.

**Elad Gil** (1:48)
Yes, Le Large, Mistral Large. They launched that. And the thing that's really, really impressed me about Mistral is just the velocity of shipping.
It's incredibly impressive. They went from basically starting the company to almost GPT-4 level in less than a year, right?

**Sarah Guo** (2:06)
Nine months, yep.

**Elad Gil** (2:07)
It's amazing.
And they have small performant models. They have Le Big Mac, the large model.
They have chat, they have multiple languages. It's just, it's very impressive execution. And then I think the other thing that they just launched or announced was that deal with Microsoft, where they're now being licensed onto Azure. And so I think the main models in Azure now are OpenAI, Lama, Mistral, and then some of the Microsoft models. So again, that's striking as well. So just very impressive progress by that company so far.

**Sarah Guo** (2:39)
I think the design space for what you actually want from models is certainly going to include state-of-the-art capability, and Mistral is very much going after that, and they've said so. But I think from the beginning, the company has talked about efficiency and latency and the ability to serve different use cases with that and also being long-term proponents of retrieval. One of the big debates in the research world right now, I don't know how much of it is a debate, but people are talking about it. I'm on one side of this, is that rag and retrieval is dead with sufficient context.
And I'm curious what you think here, but I'm more of the belief that it just opens up the set of trade-offs you can make between retrieval, more sophisticated retrieval and model reasoning by having a larger context window versus saying like, we don't need any ability to work with a specific data set versus just retrain or stuff something into context.

**Elad Gil** (3:41)
Yeah, we're going to have both in my opinion. The other thing I think that's very under discussed, and this could lead into agent stuff, but I'd like to also spend a little bit of time on Gemini before I move to agents is, if you look at a lot of the optimations that are done for areas where you had a human related sort of reasoning or other components, pre-LLM based reasoning, a lot of it was happening at inference time, right? So when you were trying to build a better poker AI, a lot of what you did was certain types of tree searches or other things when you hit inference time, right? You built the model, but at inference it did a lot of extra work.