Peter Norvig Disagrees With Yann LeCun. Here’s Why

SPEAKER_1 (0:00)

I had a pleasure talking to Peter Norvig. Thousands of students know him as the author of this monumental book, and I love it too. The funny fact is, Peter Norvig once thought ordinary businesses would never be able to use the internet at scale. Setting up a web server was too difficult. There simply were not enough people who knew how to do it. He was right about the difficulty, but he was completely wrong about how long it would last. At the AI Conference, I wanted to talk to Peter because he has watched AI change for more than 40 years and how generations of people understand it through Artificial Intelligence, A Modern Approach, which he wrote with Stuart Russell. Today, he's watching students who have never programmed, build their first apps. So what should we still be learning? What does all this new capability make possible? And where might the people who understand the technology best be getting the future wrong?

Peter Norvig (0:58)

I don't have a very good track record for prediction. So I remember when Al Gore invented the Internet, when there was an ARPANET, and I was on that as a student at Berkeley. And then Gore and others had this idea that there was going to be Internet that was open to everybody, and anybody could sign up for it, and, you know, businesses would sign up for it. And I said, that'll never work, because there's only like a couple hundred people in the world who really know how to set up a web server and do it right. How are 10 million companies going to all do that? They won't be able to do it. But I didn't realize is when there's a demand, people will make the software tools that make that process easy. So you don't have to be this Unix wizard in order to set up a network, that you just buy this package from Netscape or Microsoft and answer the questions and it works. And so I think we'll see more of that.

Maybe I'm the worst person to be asking, or us insiders who are saying, I know all the details of how you build these language models. No, you shouldn't be asking somebody like that. You should be asking an end user, what tools do you need to make your life work? I think we'll see that and we'll see this more powerful tools to do things that we used to think were hard, and then they'll become easy. Let's go back to the start of how was our book different than the previous books, because Stuart Russell, my co-author and I, we've been teaching out of the other books, and we thought the other books that are available were good, but it felt like the field has changed.

SPEAKER_1 (2:36)

It's 95?

Peter Norvig (2:37)

Yeah. Maybe over the last decade up to 95 I think maybe around 85, there was a big push away from logic and towards probability. So, Udaya Pearl was probably the leader of that, and I think everybody else saw, yes, this is the way to go, that we were too constrained by bullying logic, and AI should really be the study of uncertainty, and probability was the right way to do that. So that was one change. And then, machine learning, rather than hand coding, was another change. The move towards big data was another change. And so, we kind of felt like the older books were missing those changes to the field, and we named them The Modern Approach. And I guess all of those are still true, but now to a much bigger extreme, that sort of everything is now just scaling and having more data and more compute. So, sometimes people ask me, well, do students still need to read the first half of your book? So, the second half is on machine learning, and we covered to some extent transformers and so on. But the first half is knowledge representation and logic. And I guess my answer is, the students probably don't have to read all of that, but the LLM should definitely read all of it. And so, like if you look at AlphaGo, so the reason AlphaGo was able to beat the world champion at Go, so decades before people thought that was going to be possible, is because it used neural nets in a clever way to encode what people knew about a board position. And that would have been enough to demolish amateurs like me. But in order to beat the world champion, you have to take that and then also put in the Monte Carlo Tree Search, which is this traditional old school algorithm. And I think that's still true, that going forward we'll have this mix of saying, you need the neural nets to understand the complexity and uncertainty of the world and so on. But you also need a clever algorithm.