How Foundation Models Evolved: A PhD Journey Through AI's Breakthrough Era

**Omar Khattab** (0:00)
Nobody wants intelligence, period. I want something else, right? And that something else is always specific, or at least more specific. There is this kind of observed phenomenon where if you over-engineer intelligence, you regret it because somebody figures out a more general and maybe potentially simpler method that scales better. And a lot of the hard-coded decisions you made are things you end up regretting. So I think it's fair to assume that like models will get better and algorithms will get better. And a lot of that stuff will improve. Then the question we really ask is, intelligence is great, but what problems are you actually trying to solve? That idea that scaling model parameters and scaling, just pre-training data is all you need exists nowhere anymore. Nobody thinks that. Actually, people deny they ever thought that at this point. Now you see this massively human designed and very carefully constructed pipelines for post-training, where we really encode a lot of the things we want to do. You see massive emphasis on retrieval and web search and tool use and agent training. There is clearly a sense in which the labs have already recognized that the old playbook doesn't work. The question is, is that actually sufficient for making the best use and the most used of these language models? It's not a problem of capabilities. It's a problem of actually we don't necessarily just need models. We want systems.

**SPEAKER_2** (1:14)
The conventional wisdom says we're racing toward AGI by making language models bigger and bigger. But what if the entire framing is wrong? On today's episode, you'll hear from a16z general partner, Martin Casado and guest Omar Khattab, assistant professor at MIT and creator of DSPy. Omar doesn't think we need artificial general intelligence. He thinks we need artificial programmable intelligence, and the difference matters more than you think. Here's the paradox. Khattab has built one of the most widely used frameworks for working with LLMs, DSPy, but he's skeptical that raw model capabilities will solve our problems. While others obsess over scaling laws and parameter accounts, he's asking a more fundamental question. Even if models become infinitely scalable, infinitely capable, how do humans actually specify what they want? Natural language is too ambiguous, code is too rigid, we need something in between, a new abstraction layer that lets us declare intent without drowning implementation details. Think of it as a jump from assembly to C, but for AI systems. The stakes are higher than prompt engineering. This is about whether AI becomes a programmable tool we can reason about and compose, or just an inscrutable oracle we prompt and pray. We get into the three irreducible pieces of an AI system. Why the model god is a dead end, and what it actually means to build software when intelligence is cheap but the specification is hard.

**Martin Casado** (2:37)
Well, listen, Omar, it's great to have you, and congratulations on everything. Just so for everybody that's listening, Omar is doing some, in my opinion, the more interesting technical work in building frameworks around LLMs and models, and a lot of this has consequences on things like AGI and capabilities and everything else, and a lot of your comments on social media to me have been some of the most insightful, so I've been really looking forward to having you on the podcast.

**Omar Khattab** (3:04)
Thank you for hosting me, Martin. I'm going to go to chat as well.

**Martin Casado** (3:06)
Awesome. Listen, maybe let's just start with your background, since we have some shared roots, and then we'll go from there to a general conversation.

**Omar Khattab** (3:16)
So I mean, I'm now an assistant professor at MIT. I started a few months ago in electrical engineering and computer science. I'm part of CSAIL. I did my PhD at Stanford, where I think the timing was really interesting. I started in 2019 and I graduated about a year ago. That timing was really great because foundation models as a concept, didn't even necessarily have that name. We hadn't coined it at Stanford yet. It was starting to take shape. BERT was around for about a year at the time. But people hadn't really figured out how to make them work. But I would say as importantly, how to make use of them to build different types of systems and applications, which is basically what I did throughout my whole PhD.

**Martin Casado** (3:55)
I presume you're the primary person behind DSPy, is that correct?

**Omar Khattab** (4:00)
You could say that, yeah.

**Martin Casado** (4:02)
Yeah. For those of you who don't know DSPy, is why the user are going to be talking about it. That's one of the most widely used, I would say, open source projects around prompt optimization for LLMs. So maybe let's just go ahead and start. You have tweeted about whether LLMs will get to AGI or not. I know it's a kind of very fluffy, high level place to start, but we'd love to hear your thoughts on, are we headed towards AGI in the near term? Is this an app's goal? Like, where do you weigh down? And it is particularly timely right now, given the conversation that Andrei Karpathy just had on the Dorkish Podcast, where he was like, well, you know, maybe 10 years if you're optimistic. Where do you weigh in on this debate?