One Brain, Any Body: Google DeepMind's Keerthana on Gemini Robotics 2, Cross-Embodiment & Humanoids

Nathan Labenz (0:00)

Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to welcome Keerthana Gopalakrishnan, staff research scientist at Google DeepMind and research lead for Gemini Robotics, back for her fourth annual appearance on the show. One of the biggest questions in AI today is how soon will general purpose robots become broadly useful? AI is already affecting the world in major ways, even in its purely digital form. But the most sci-fi forecasts for the AI future predict that robotics will soon hit key tipping points. AI 2027, for example, predicts that humanoid robots will become useful sometime in mid-2027, and that by 2028, a whole robot economy could take shape, where robots build more robot factories, which in turn produce more robots, leading to unprecedented exponential economic growth, but also creating serious risk of AI takeover.

So, how does that vision line up with the reality of robotics research today? We begin with a discussion of the extremely viral Robot Olympics held in China this summer, as I was really curious to get Keerthana's take on what it means that humanoid robots can now run faster than the fastest humans. Her point of view, as you'll hear, inverts the usual US-China dichotomy in AI. She says that while the videos are obviously impressive, skills like running on a flat track are actually relatively easy to train in simulation. And more to the point, foot speed is not really a limiting factor in the utility that robots can provide. As such, her team at Google is more focused on practical value.

From there, we get into Gemini Robotics 2, a suite of three models that Google released this summer. Gemini Robotics ER 2 for Embodied Reasoning, which Keerthana describes as a System 2 for Robotics Control, is based on Gemini Flash and is available via the API, allowing developers to define the affordances available to the models as tools, just like we do with digital agents. I played around with it and found it remarkably accessible. The other two models, Gemini Robotics 2 and Gemini Robotics On Device 2, translate these higher level tool calls down to the level of robot action, and are now capable of controlling the whole robot from fingertips to feet on a wide range of form factors, but are currently available only to trusted testers. Having understood how these models work, we again zoom out, and I asked Keerthana how she understands progress in robotics overall. On the one hand, like so many other AI researchers recently, she says that she has been surprised by the pace of progress. But at the same time, she still feels that robotics remains in its GPT-2 era. We have seen a number of recent demos of robots learning new tasks from just a few or even just a single human demonstration. But Keerthana argues that the range of tasks you can teach this way, and the generalization profile of robotics models across different robot bodies, still isn't strong enough to deliver the versatility and reliability that real-world use cases demand.

Toward the end, we move on to talk about recent progress in hardware, where robots are likely to be deployed at scale first, whether capabilities or safety, alignment and robustness will ultimately be the limiting factor for the consumer market, and we consider the future of the field, including whether we can continue to run LLM-derived robotics models in a loop as today's systems mostly do, or we'll need some sort of predictive world modeling, as we humans use, to create systems that can anticipate and smoothly interact with the world.

On that question and on the timeline for key utility milestones, Keerthana isn't one to speculate too wildly. For her, these are all empirical questions to be resolved by research and experimentation. But what is clear to me coming out of this conversation is that robotics will either need to hit key tipping points soon, or some of the most aggressive timelines for AI transformation will have to be pushed back.

With that, I hope you enjoy this very well grounded update on the state of robotics with Keerthana Gopalakrishnan from Google DeepMind. Keerthana Gopalakrishnan, staff research scientist at DeepMind and research lead for Gemini Robotics. Welcome back to The Cognitive Revolution.

Keerthana Gopalakrishnan (4:31)

Hey, nice to be here.

Nathan Labenz (4:33)

Great to see you again.

It's been about a year, obviously a lot has happened. I wanted to start off with something fun. There was recently a Robotics Olympics in China, and clips were obviously flying all over the internet and people were having a laugh and at times, I think a little empathy for robots, which is something we've talked about in the past. I'd have to just start by getting your observations and takeaways, given the depth of knowledge that you have on the subject. What stood out to you in watching clips of the Robot Olympics?