Jagdeep Singh: Teaching Robots From Internet Video at Rhoda AI

Jagdeep Singh (0:00)

Despite all the billions that have been raised, despite all the cool videos that have been published, I actually can't think of a single example of a generalist, autonomous robot that's doing useful work in the field today.

Coming from outside the space, as I ramped up on understanding what the issues were, why robotics hadn't seen the same kind of progress that other AI models had seen, it became clear that the central issue was the lack of data. The dominant paradigm in robotics is the VLA, the Vision Language Action Model. Obviously, we have a different approach. Our belief is that while VLA's are very helpful, they too have a ceiling.

Brian Heater (0:36)

You're working with a lot of these existing customers in a lot of cases with off-the-shelf hardware. So why ultimately is the company developing its own hardware?

Jagdeep Singh (0:46)

We played with a bunch of third-party hardware and found these issues with payload capacity, reliability, safety and dexterity. So those are the four key things that are missing from today's humanoids.

Brian Heater (0:55)

And you get the sense that the partners that you're working with right now are eventually going to move towards a more general purpose hardware?

Jagdeep Singh (1:02)

Oh, absolutely. The tasks that are left that are not automated are tasks that really can't be well automated through traditional means. And those tasks really require the generality of the humanoid. The more immediate question, I think, for the industry is, do you have an AI model that can just deal with the messiness of the real world?

Brian Heater (1:43)

Hello and welcome to another episode of Automated. I'm Brian Heater, the managing editor at the Association for Advancing Innovation, thrilled this week to take another deep dive into one of the bleeding edge pioneers of physical AI. Jagdeep Singh is the CEO and founder of Rhoda AI, a startup that has been showcasing some extremely impressive and promising progress in robotic learning. We go really deep on this one. Please enjoy. Thanks as always for listening. If you've been enjoying the show, it would be a huge help if you would like and subscribe, tell a couple of friends, and while you're at it, why not like and check out and subscribe to the newsletter over at automated.fm. Now, please enjoy this conversation with Jagdeep Singh of Rhoda AI.

So the biggest thing I'm always curious about in the life of the serial founder is when it's clear to make the jump from this current company that you built and are running into this entirely new venture.

Jagdeep Singh (2:48)

Yeah, I don't think there's a formula for that. I think there's a lot of factors that go into it. In my particular case, the last two companies I've run, I ran for on the order of 10 or more years. Both of those we took public. And at some point, running a public company gets a bit fatiguing. And it's time for me to step down and let the next level of leadership take over.

But I'm always keeping track of new technologies. And so there was always an interest in new disruptions that are around the corner. And so if the time that it's right for me to step down from a company coincides with a new disruption that's around the corner, then everything comes together and it's kind of a perfect storm.

Brian Heater (3:42)

Yeah, one of the things that I suspect a lot of people are wondering who follow the industry is what is it precisely that's happened in the last, I guess, you know, 24 months, you know, two to three years that has caused all of these physical AI companies, including Rhoda, to like suddenly start coming out of the woodwork right around the same time.

Jagdeep Singh (4:09)

Well, I think obviously the biggest change is that, you know, the underlying AI technology has really advanced to the point where you can do more useful things now, right? So robotics used to be an area where you had a modular approach. There was a perception module and a planning module and a control module and so on.

And with the emergence of end-to-end AI models that learn purely based on data and even things like CNNs, which at one point were helpful, really introduce a predefined bias into how the problem is solved. And a more end-to-end approach feeds in data into the network. The network learns what it needs to learn and then produces the output. And those approaches allow for a higher ceiling on what the models can achieve, right? So if you, the classic, I guess, you know, curve that you look at here is, you know, if you are data constrained or compute constrained, then having human biases, you know, inductive biases inserted into the model itself can be helpful because you can do more with data or less compute. But those inductive biases also introduce ceilings on how good the models can get. And if you go to a more end-to-end approach, you need more compute and more data to get useful results. But the model might be able to learn things that go beyond what those inductive biases can provide. And so what's happened in robotics is that with the increased amount of compute available, and of course, the sophistication of models, not only things like attention, which is obviously even critical, but a lot of progress in the area of diffusion, you've been able to process data end to end, and that's enabled a way to do robotics that doesn't require the traditional modular approach. And I think that's been the key. So if you look at, obviously, the prevailing paradigm, the dominant paradigm in robotics is the VLA, the Vision Language Action Model. And what they do, of course, is they start with a pre-trained language model or vision language model, which gives them a good prior on language. And then they post-train it using some robot tele-operation data. And that tele-operation data is, that training happens end to end. So the models can learn to do pretty useful things.