Eric Jang – Building AlphaGo from scratch

**Dwarkesh Patel** (0:00)
Today, I'm here with Eric Jang, who was most recently Vice President of AI at One X Technologies, before that Senior Research Scientist at what is now Google DeepMind Robotics. And you've been on sabbatical for the last few months. One of the things you've been doing is rebuilding and improving and hacking on AlphaGo. And so today, what we're going to do is you're going to explain building AlphaGo from scratch and what it tells us about the future of AI research and development. But before we get to that, why is AlphaGo interesting?
Why is this the project you decided to do on sabbatical rather than just hang out at the beach?

**Eric Jang** (0:33)
Sure, yeah.
I like making things. And AlphaGo and GoAI is one of those things that really got me into the field. When I saw the kind of early breakthroughs on AlphaGo in 2014, 2015, 2016 and so forth, it was just profound to see how smart AI systems could become and the kind of computational complexity class that they could tackle with deep learning. This is a problem that has long been understood to be kind of intractable for search and yet it was solved through deep learning. And so that was quite mysterious to me and I've always wanted to understand that phenomena a little bit better. My training is often in deep neural nets for robotics where the decisions made by the neural networks are a bit more intuitive, but AlphaGo is a sort of problem where the decisions are actually the result of a very, very deep search. And it's always been very mysterious to me how like a 10-layer network can sort of amortize the simulation of something so deep in the game tree.

**Dwarkesh Patel** (1:34)
Yeah, interesting.

**Eric Jang** (1:35)
So if you plot out how much compute it took to build various iterations of strong GoBots over the years, you can see that in 2020, there was a open-source project called Catago by David Wu from Jane Street who basically achieved a 40x reduction in compute needed to train a really strong GoBot Tabula Rasa. I'm not certain if it's stronger than AlphaGo or Alpha Mu 0, but it's very, very strong and this is what most Go practitioners today train against when they're playing in AI.
Thanks to LLM coding, what took a whole team of research scientists at DeepMind and millions of dollars of research and compute can now be done for a few thousand dollars of rendered compute.

**Dwarkesh Patel** (2:14)
By the way, if you're listening to this on an audio platform, this is a Blackboard lecture, I highly recommend switching over to a video platform like YouTube, if you can, to look at the math and the graphs and the Go board. Okay, I guess we should first discuss how Go works.

**Eric Jang** (2:28)
Great.

**Dwarkesh Patel** (2:28)
So yeah, how does the game work?

**Eric Jang** (2:31)
So the game of Go is a very simple one that can be implemented quickly and easily in a computer.
The objective of the game is basically to put down black and white stones and try to occupy as much territory in the game as possible. So I might start by putting down a black stone. Black always goes first. So go ahead. So the way you capture and oppose stones is that for every intersection, if you can surround all four of its neighbors with your stones, then this one is sort of cut off from oxygen, if you will, and then it is a dead stone. So then now I control these four stones as well as this empty intersection here. So there's like slight variations between Chinese, Japanese and what is called Trump-Taylor rules.
Trump-Taylor rules are designed to be completely unambiguous for Go, so this is what all Go AIs train against and resolve against. So in typical Go, like when humans play, you're actually not allowed to put this white stone down here. It would be instant suicide. In Trump-Taylor, it's actually fine. You put it down and then it immediately resolves to death. So the outcome is sort of the same. Let's go ahead and start over and play a few stones and then I'll explain some. So I'll just start there.

**Dwarkesh Patel** (3:44)
I'm basically playing randomly here, but I'm trying to get around your stones and see if I can close them.

**Eric Jang** (3:57)
So this move basically exposes one empty neighbor for your white stone, and it's very akin to a check in chess, where if you don't respond immediately by putting in one here, then I can immediately capture this.

**Dwarkesh Patel** (4:09)
I see, okay. Because it is the diagonals that determine whether it rounded in.