Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI

**SPEAKER_1** (0:00)
Today, I have the honor of chatting with Jeff Dean and Noam Shazeer. Jeff is Google's chief scientist, and through his 25 years at the company, he has worked on basically the most transformative systems in modern computing from MapReduce, BigTable, TensorFlow, AlphaChip. Genuinely, the list doesn't end. Gemini now. And Noam is the single person most responsible for the current AI revolution. He has been the inventor or the co-inventor of all the main architectures and techniques that are used for modern LLMs from the transformer itself to make sure of experts, and to mesh tensorflow, to many other things. And they are two of the three co-leads of Gemini at Google DeepMind. Awesome. Thanks so much for coming on.

**SPEAKER_2** (0:48)
Thanks for having us.

**SPEAKER_3** (0:49)
Super excited to be here.

**SPEAKER_1** (0:50)
Okay, first question. Both of you have been Google for 25 or close to 25 years. At some point early on in the company, you probably understood how everything worked. When did that stop being the case? Do you feel like there was a clear moment that happened?

**SPEAKER_2** (1:06)
I mean, I know I joined, and like at that point, this was like end of 2000, and they had this thing, everybody gets a mentor. And, you know, so, you know, I knew nothing. I would just ask my mentor everything, and my mentor knew everything. It turned out my mentor was Jeff. And it was not the case that everyone at Google knew everything. It was just the case that Jeff knew everything, because he has basically written everything.

**SPEAKER_3** (1:31)
You're very kind. I mean, I think as companies grow, you kind of go through these phases. Like when I joined, you know, we were 25 people, 26 people, something like that. And so you eventually learned everyone's name, and even though we were growing, you kept track of all the people who were joining. At some point, then you kind of lose track of everyone's name in the company, but you still know everyone working on, you know, software engineering things. Then you sort of lose track of, you know, all the names of people in the software engineering group, but you know, you at least know all the different projects that everyone's working on. And then at some point, the company gets big enough that, you know, you get an email that Project Platypus is launching on Friday, and you're like, what the heck is Project Platypus?

**SPEAKER_2** (2:13)
Usually, it's a very good surprise. You're like, wow, Project Platypus. I had no idea we were doing that. And it turns out brilliant.

**SPEAKER_3** (2:21)
It is good to keep track of what's going on in the company, even at a very high level, even if you don't know every last detail. And it's good to know lots of people throughout the company so that you can go ask someone for more details or figure out who to talk to. I think with one level of indirection, you can usually find the right person in the company if you have a good network of people that you've built up over time.

**SPEAKER_1** (2:42)
How did Google recruit you, by the way?

**SPEAKER_3** (2:45)
I kind of reached out to them, actually.

**SPEAKER_1** (2:48)
And Noam, how did you get recruited?

**SPEAKER_2** (2:50)
What was it that you did that? I actually saw Google at a job fair in like 1999 And I assumed that it was like already this huge company that had no point in joining. Because everyone I knew used Google, I guess that was because I was a grad student at Berkeley at the time. I guess I've dropped out of grad programs a few times.
But it turns out that actually it wasn't really that large. So it turns out I did not apply in 1999, but just kind of sent them a resume on a whim in 2000 because I figured it was like my favorite search engine and figured I should apply to multiple places for a job. But then, yeah, it turned out to be really, really fun. Looked like a bunch of smart people doing good stuff. And they had this really nice crayon chart on the wall of the daily number of search queries that somebody had just been maintaining and looked very exponential. These guys are going to be very successful. And it looks like they have a lot of good problems to work on. So it's like, okay, maybe I'll go work there for a little while and then have enough money to just go work on AI for as long as I want after that.

**SPEAKER_1** (4:07)
In a way, you did that, right?

**SPEAKER_2** (4:08)