What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang

**Percy Liang** (0:07)
For ages, human level has been the target for AI, and that has really been kind of a north star that has fueled many dreams and efforts and so on over the decades. But I think we're getting to a point where along many axes, it's superhuman or should be superhuman, and I think we should maybe define more of an objective measure of what we actually want.
This is more of a general statement about how we should think about technology, but not just chasing after mimicking a human because we have a lot of humans.

**Sarah Guo** (0:45)
This is the No Priors podcast. I'm Sarah Guo.
We invest in, advise, and help start technology companies.

**Elad Gil** (0:52)
In this podcast, we're talking with the leading founders and researchers in AI about the biggest questions.

**SPEAKER_4** (1:04)
We're very pleased today to have Dr. Percy Liang, Professor of Computer Science here at Stanford and Director of the Center for Research on Foundation Models, a recently founded center here. Dr. Liang is the author of over a hundred heavily cited research papers around helping machines understand natural language, helping humans reason about those models, and has contributed a number of novel technical approaches and creative uses of data to the machine learning field.
As a special treat, we're recording here in his office at Stanford today. Thanks, Percy.

**Percy Liang** (1:36)
Great. Welcome.

**SPEAKER_4** (1:38)
I think just to start, can you tell us a little bit about how you got into the machine learning research field in your personal background?

**Percy Liang** (1:46)
I've been in the field of machine learning and natural language processing for over 20 years. I started getting into it in undergrad. I was undergrad at MIT. I liked theory. I had a fascination with languages. I was fascinated by how humans could just be exposed to just strings of text, speech, and somehow acquire very sophisticated understanding of the world and also syntax and learn that in a fairly unsupervised way.
My dream was to get computers to do the same. So then I went to grad school at Berkeley, and then after that started at Stanford. Ever since, I've been in pursuit of developing systems that can really truly understand natural language.
Of course, in the last four years, this once upon a time dream has really taken off in a sense. Maybe in a not a way that I would necessarily expect, but with the coming out of large language models such as GPT-3, it's truly astonishing how much of the structure of language and the world that these models can capture.
In some ways, it harkens back when I actually first started in NLP, I was training language models, but of a very different type. It was based on hidden Markov models, and there the goal was to discover hidden structure in text. I was very excited by the fact that it could learn about, tease apart what words were like city names versus days of the week and so on. But now it's on a completely different level.

**SPEAKER_4** (3:22)
You've worked on multiple generations of NLP at this point, pushing the forefront of semantic parsing. Was there a moment at which you decided that you were going to focus on foundation models and large language models?

**Percy Liang** (3:34)
Yeah, there was a very decisive moment, and that moment was when GPT-3 came out. That was in the middle of the pandemic, and it wasn't so much the capabilities of the model that shocked me, but it was the way that model was trained, which was basically taking a massive amount of text, and asking the model to predict the next word over and over again, billions of times.
What rose from it was not only a model that could generate fluent text, but also a model that could do in-context learning, which means that you can prompt the language model with instructions.
For example, summarize this document, give it some examples, and have the model on the fly in context, figure out what the task was. And this was a paradigm shift, in my opinion, because it changed the way that we conceptualize Machine Learning and NLP systems from these bespoke systems where it's trained to do question answering, to trained to do this, to just a general substrate where you can ask the model to do various things. And the idea of a task which is so central to AI, I think, begins to dissolve. And I find that extremely exciting. And that's the reason later in 2021, we founded the Center for Research on Foundation Models. We coined the term foundation models because we thought there was something that was happening in the world that somehow large language models didn't really capture the significance. And it was not just about language, it was about images and multimodality. It was a more general phenomenon. And we coined the term foundation models and then the center started. And it's been sort of a kind of a roller coaster ride ever since.