Commoditizing the Petaflop — with George Hotz of the tiny corp

**Swyx** (0:10)
Hey, everyone. Welcome to the Latent Space Podcast. This is Sviks, Ryzen editor of Latent Space, and Alessio is taking over with the intros. Alessio's partner, NCT One Residence and Despo Partners.

**Alessio** (0:20)
Hey, everyone. Today we have GeoHot on the podcast, aka George Hotz for the human name.
Everybody knows George, so I'm not going to do a big intro. A couple of things that people might have missed. So you traded the first ever unlocked iPhone for a Nissan 350Z and three new iPhones. You were then one of the first people to break into the PS3 around arbitrary code, you got sued by Sony, you wrote a rap song to fight against that, which is still live on YouTube, which we're going to have on the show notes.
Then not go to Tesla to build vision and say you started Comma.ai, which was an amazing engineering feat in itself until you get a season disease from the government to not put these things on the street. Turn that into a research only project.

**George Hotz** (1:00)
You know they're out there.

**Alessio** (1:01)
Yeah, no, they're out there. But you market them as a research kind of like no warranty.

**George Hotz** (1:06)
Because I use the word DevKit, that's not about the government. That's nothing to do with the government. We offer a great one year warranty.
The truth about that is it's gatekeeping. What's the difference between a DevKit and not a DevKit?
Nothing. Just the question of do you think it's for you? If you think it's for you, buy it. It's a consumer product. We call it a DevKit. If you have a problem with that, it's not for you.

**Alessio** (1:28)
That's great insight.
I was going through your blog post to get to the day. You've wrote this post about the hero's journey. You linked this thing called the portal story, which is the set of stories in movies and books about people living this arbitrary life, and then they run to this magic portals, takes them into a new very exciting life and dimension.
When you've wrote that post, you talked about Tinygrad, which is one of the projects we're working on today. You mentioned this is more of a hobby, something that is not going to change the course of history. Obviously, you're now going full speed into it. So we would love to learn more about what was the portal that you run in to get here.

**George Hotz** (2:03)
Well, what you realize is, you know what made me realize that I absolutely had to do the company? Seeing Sam O'Lan going in front of Congress.
Why? What are the odds they nationalize in video? What are the odds that large organizations in the government, but of course, I repeat myself, decide to try to clamp down on accessibility of ML compute? I want to make sure that can't happen structurally. So that's why I realized that it's really important that I do this. And actually, from a more practical perspective, I'm working with NVIDIA and Qualcomm to buy chips. NVIDIA has the best training chips, Qualcomm has the best inference chips.
Working with these companies is really difficult. So I'd like to start another organization that eventually in the limit, either works with people to make chips or makes chips itself and makes them available to anybody.

**Alessio** (2:48)
You share kind of three core thesis to Tinycorp. Maybe we can dive into each of them. So XLA Prime Torch, those are the complex instruction system. TinyGrad is the restricted instruction system. So you're kind of focused on, again, TinyGrad being small, not being over complicated and trying to get as close to like the DSP as possible in a way, where it's at more.

**George Hotz** (3:08)
Well, it's a very clear analogy from how processors developed. So a lot of processors back in the day were CISC, complex instruction set, System 360, and then x86.
Then this isn't how things stayed. They went to now the most common processors arm, and people are excited about RISC-V.
RISC-V is even less complex than R. No one is excited about CISC processors anymore. They're excited about reduced instruction set processors. So TinyGrad is we're going to make a RISC op-set for all ML models. Yeah, it can run all ML models with basically 25 instead of the 250 of XLA or PrimTorch. So about 10x less complex.

**Alessio** (3:48)
You talked a lot about existing AI chips. You said if you can write a fast ML framework for GPUs, you just can write one for your own chips. So that's another one of your core insights. I don't know if you want to expand on that.