Pete Florence: When Robots Improvise

Pete Florence (0:00)

2011 Fukushima nuclear disaster. It's the Fukushima number one nuclear power plant. Japan has been hit by its biggest earthquake. Simply heartbreak. There wasn't a robot in the world that could walk into the nuclear facility after the meltdown and just turn a valve. And so we had to send people in for that. If you're not touching and moving the world, and you're a robot, what are you doing? Any process in the world that is gated by physical progress, building houses, whatever it is, we could just do that way faster, way better. Gem 1.5, it has a couple different capabilities where you can learn a physical task in just a single demonstration. The success rates of all these are modest today, but the fact that these success rates are anything above zero is the incredible thing.

Mario Gabriele (0:40)

Shulto Douglas, well-known anthropic researcher, called this sort of a GPT-3 moment for robotics.

Pete Florence (0:46)

The example that we highlighted is using a brush to sweep an object into a bowl, and then instead you take the brush away and you give it a dustpan.

Mario Gabriele (0:53)

One of the stories of these past few years has been finding sort of the killer use case in coding. When it comes to physical intelligence, do you have an intuition of what that killer use case might be?

Pete Florence (1:03)

I think the best analogy would be dexterity. If we can really crack dexterity, then the robots can build the robots and the robots can start building the next generation of robots.

Mario Gabriele (1:17)

Robots don't learn the same way that people do. Teaching a robot a new task can take months of programming, data collection and retraining. Generalist, an AI company for robotics, is compressing that process down to seconds. In one of its recent demonstrations, a robot was shown just one time how to take money out of a wallet. It then completed that task immediately without needing to be retrained. Today's guest, Pete Florence, is Generalist's co-founder and CEO. The company is building what it calls general intelligence for the physical world. AI models meant to work across different types of robots, from factory arms to humanoids.

Since its founding a little over two years ago, Generalist has grown into a key player in the robotics race. In this episode, Pete and I discuss one-shot learning, why dexterity remains so hard, where physical AI follows the same scaling laws as digital AI, and which industries change first when robots learn new skills much faster. I'm Mario, and this is The Generalist.

Well, I think the only place to start is that I think you have the best name possible.

Pete Florence (2:31)

Likewise, Mario.

Mario Gabriele (2:33)

Very good taste. Well, I'm really looking forward to this conversation and getting into what you're doing at Generalist. It feels like a particularly exciting time. Earlier this month, I think it was mid-August, you released Gen 1.5, which is your latest foundation model, and it seemed to attract a lot of excitement.

I saw Sholto Douglas, well-known anthropic researcher, called this a GPT-3 moment for robotics. Maybe you can tell us a little bit about what Gen 1.5 is and why it matters.

Pete Florence (3:04)

Sure.

Gen 1.5 is definitely in a few different ways, like a type of model that we had been waiting for for a while, at least in terms of glimpses of types of capabilities that both are just exciting in and of themselves and then also point towards what might be a future of what's coming next. So overall, Gen 1.5, it has a couple of different capabilities in what we call one-shot or a few-shot learning, and also some other things in what we think are really remarkable levels of zero-shot physical generalization. These one-shot and few-shot broad types of learning capabilities where you can learn a physical task in just a single demonstration. And mind you, just to be clear, the success rates of all these are modest today, but the fact that these success rates are anything above zero is the incredible thing. And if I dial back a few years, one of my co-founders, Andy Zhang, and myself, we've been talking for a while about this threshold of crossing into a very generalized one-shot and few-shot learning capability in robotics. And of course, the rise of large language models around 2020, yes, of course, GPT-3, that was the main thing that, or at least arguably the main thing that people were especially excited about with the model GPT-3, was this broad one-shot and few-shot learning capability. They coined the term, in context learning, in that paper, it's been an incredible thing to see.