Chelsea Finn: This is the State of the Art in Robotics

**Chelsea Finn** (0:06)
Everyone, today, I'm going to be talking about the state of the art of physical intelligence. And in particular, two years ago, I founded a company called Physical Intelligence. And we're really interested in how we can basically develop any robot or allow any robot to do any task in the real world. And I was actually spoke at this event a year ago, last year. And at the event last year, I shared some of our progress at the company at Physical Intelligence, where we could do things, really complicated tasks like folding, unloading and folding laundry. And I also talked about how for the first time, we showed how robots can do useful tasks in environments and rooms they've never been in before. Now, since then, since one year ago, we have gotten robots to do a lot of other really cool things. So for example, we've gotten robots to be able to wash a greasy pan in the top right, or peel a carrot in the video below that, or make a grilled cheese sandwich in the video below that, or slice a zucchini and so forth. But what I'd really like to focus on today isn't cool videos of robots doing lots of different things, but what it actually takes to get robots to be useful in the real world. Specifically, how can we develop general purpose robots that are useful in the real world? Now, there are two aspects of this. The first is general purpose, how we can develop general purpose models, and the second is actually bringing in those models to the real world, so that they can actually have an impact and be useful to people. And in the first part, I'll talk about being useful in the real world. So to actually bring a technology to the real world, I think we need to figure out, it's helpful to actually look at what people have done in the past to bring AI into the real world. And if we look at a timeline of major production launches that are leveraging technology like machine learning, we can see a timeline like this. I think the really the first early examples of machine learning being used for real, in the real world, were for things like product recommendations and ad ranking. And then five years later, we started to see not just machine learning being used, but deep learning being used for the same sorts of applications. This was a really exciting advance because deep learning is an algorithm that actually isn't like you can really apply it out of the box to scenarios that involve really complex inputs and outputs and so forth. And it makes it easier to translate to other applications. But from there, I think that even more exciting kind of moment in time that we saw in terms of machine learning and AI production was in 2022 with the launch of ChatGBT. And this was the first time where we saw a general purpose model truly being used by many different people in the real world. Within five days, ChatGBT had reached a million users. And then of course, more recently, we've seen things like cloud code also be incredibly useful, hopefully to many of us in the real world and other coding agents. Now, if we look at how AI has been used in the real world and kind of look at this, I think there's a few different takeaways we could make. The first is that generalist models are increasingly being used for real world problems. So we're actually seeing general purpose, like generalist AI models that can do many, many different things actually be used in the real world, and we see that transition from the left to the right. But I also think that there's a more nuanced observation that we can make from looking at these applications. In particular, if we look at all of these different applications that are used where machine learning has actually been useful in the real world, and actually been profitable and so forth. But in all of these applications, the customer is making a decision based off of the recommendation of the AI model, more or less. And this means that if the customer is ultimately making the decision, this means that if the system makes a mistake, that's okay, because usually the person can recognize that or decide what to do even despite that mistake. So even when these sorts of systems aren't perfect, they're still incredibly useful to different people, and there's less pressure on them to be completely perfect. And I think that actually physical AI and robotics is pretty different from this. Where if we think about physical AI that are actually operating in the physical world, they have to be directly making decisions that affect the physical world. And this means that they're going to be far more useful when they're operating fully autonomously. And as a result, this requires us to develop physical AI systems that make far fewer mistakes than the machine learning systems that have been deployed thus far. Now, one really exciting thing to kind of highlight that has happened recently is a year ago Waymo passed the quarter of a million weekly autonomous rides, suggesting that it is really possible to develop a machine learning based system that can operate in a trustworthy and autonomous way directly in the physical world. And I think that brings a lot of hope and optimism for actually doing the same with the rest of AI in the physical world. So if we want to develop general purpose robots in the real world, I think we need to think about how we're going to make them autonomous for long periods of time so that they're actually useful rather than having them rather than something where a human is basing decisions on the predictions of the model.