Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile

Walter Goodwin (0:00)

Right now, we're trying to build a single chip. If you crack open an Nvidia system, it has anywhere between kind of six and nine custom chips, all built by Nvidia, to come together to build something that is really, really potent. So there's already a gap there. You know, it'd be great to be productive enough that we could start to build our own responses in that same kind of way. This idea of being able to have at any given moment a kind of rolling frontier of bets that you hope are deeply aligned, in which you're ready to trigger the ramp of, this is a huge advantage if you can kind of build that machine and build that engine. It's the equivalent of the frontier model for the chip space, is if you can just find a way to structurally carve out a three to six months advantage, you will be winning all of those deployments.

Sarah Guo (0:46)

Hi listeners, welcome back to No Priors. Today I'm here with Walter Goodwin, founder and CEO of full stack AI chip company Fractile. We talk about what it means to be full stack as a company, the technical bets they're making, why he's so focused on memory bandwidth, his predictions for frontier model architectures in the future and the structure of the chip market at the frontier today and tomorrow, amongst Nvidia, AMD, internal efforts and this new class of accelerators. Welcome, Walter. Walter, thanks so much for doing this.

Walter Goodwin (1:18)

Very good to be here, Sarah.

Sarah Guo (1:19)

You started this incredibly interesting company called Fractile. Can you give us a overview of what you guys do?

Walter Goodwin (1:24)

Fractile is a chip company. We build very, very fast inference chips for the world's largest models. This sort of bet on speed is something which we've had right from the outset. Started the company in summer of 2022 I think at the time we were starting to see two things. One was obviously the arrival of foundation models trained on the Internet and generalizing across everything. The other was a few kind of wise people saying, we need to find a way to pour more compute into these models at test time. We need to find a way to take what we had for kind of AlphaGo, where you have a capable neural network, but then it becomes superhuman as you roll it out at scale, and bring that to kind of language modeling, for example. So our big pursuit over the past four years has been to find a way to build chips that will allow for kind of simultaneously taking these huge models and running them much, much, much faster than the chips of today.

But do that in a way that kind of scales, scales to models beyond the frontier today, scales to extraordinarily long contexts, and so that's the mission that we've been on.

Sarah Guo (2:29)

It's become a much more interesting chip landscape in the four years since you started the company. Where would you place yourself in the overall GPU and accelerator landscape for inference?

Walter Goodwin (2:41)

Yeah, there's a kind of extraordinary zoo now of options available. If you look across this kind of entire space of kind of AI ASICs, one of the things that I think is very striking is there's a lot of relatively identikit chips out there. And this is kind of a structural, it's a kind of structural industry property where if you look especially across kind of hyperscaler efforts, and Google kind of kicked this off with the TPU more than 10 years ago, you see this paradigm where although there are a number of kind of proprietary internal chips, you know, Google with the TPU, Meta with MTIA, Microsoft with Maya, OpenAI now with Jalapeno.

These chips that are ultimately developed and delivered in partnership with kind of a relatively small number of what we might call ASIC houses, these kind of back-end delivery houses, Broadcom is the largest, you know, $2 trillion company that help people kind of realize these designs. And so when you kind of zoom into this, sort of seeming embarrassment of riches in the spectrum of what exists today, you see that actually there's a lot of similarity across these platforms. You know, all of these chips, you have HBM, this type of DRAM memory that is, you know, common to Nvidia GPUs, AMD GPUs and all of these other ASICs. You have the same kinds of bets on tensor cores to do your matrix multiplications, the same advanced packaging with TSMC. And so I think one of the things that we see in the landscape is there's a relative dearth still of kind of efforts that go all the way up and down the silicon stack and try and build kind of fundamentally new capabilities. And this is kind of a structural thing. There's just not that many teams that have decided we're gonna build from, you know, what you might call the kind of, you know, the architecture layer, the front-end design layer, which mostly looks like kind of writing code, but also all the way down to what we call kind of physical design, the process technology, the kind of foundry interactions, the stuff that essentially has you flying back and forth to Taiwan or Korea every week.