Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

swyx (0:00)

All right, we're here on-site with Jason Ma, the co-founder and chief scientist of Dyna Robotics. Thanks for having us in your...

Jason Ma (0:06)

Of course, thank you for coming.

Vibhu (0:07)

Yeah, it's been a blast. We got to see, you'll see a lot of V-roll, but we got to see drinks being pulled, a lot of evals, robots with knives for the first time.

So under the hood of all this, you have a world action model. What is that?

Jason Ma (0:22)

Yeah, so last month, we released our world action model called Dyna-2.

So it's the first world action model trained on more than one million hours of data.

So going back to your question for this world action model, it's a model that's able to predict the future video, the future state of the world. So hence the name world model. But in addition, you can also use that future generation to also produce actions robots can execute in the physical world to control the robot. So why do we do that? It's a very powerful architecture. Because if you think about how humans act in the physical world, in our brain, we're implicitly thinking about how the world will evolve based on actions. So now we have a model architecture that's able to do that to guide robot actions. So what we find is that by doing so, the model is much more dexterous, much more robust, and much more capable of generalizing to environments to scenarios it's never seen before.

Vibhu (1:16)

Yeah. I want to prompt this question a bit. I think a lot of people hear world model and they think you're outputting video or world or 3D state.

You're also doing action. I think most people don't understand what models are under the hood of robotics. What is input, output? What's the training data? What is the architecture?

Give us all the details.

Jason Ma (1:39)

Yeah.

For inputs, the most important input is video.

You can think about videos that the robot is capturing with its cameras. Video or images of the past many seconds, that's the input. But for robots, what's really interesting is that robot also has a lot of actuator, a lot of joint encoders on itself. So the robot also knows the angles of its joints. So those can also be inputs to this world model. That's something that's very different from just a pure video model that's only looking at videos, right? So that's the input. Additionally, for robots, you want to be able to talk to it, to instruct it to do different tasks, right? So you can also feed it like language instruction. So those are what I consider to the basic set of inputs that go into any robotics models including world action models. And once we have that, the output at the basic level, it needs to output actions the robots can execute on the hardware. But in the case of world action model, what's really interesting is that in addition to the actions, you can also output future video, right? So basically, you can almost output the future video or some representation of it first and then use that to guide robot actions. So the way I think about it is like, this is kind of analogous to, you know, chain of thought, you know, basically the robots, the models are reasoning in the visual, the physics space and then outputting actions. So that's the key difference between world action models outputs versus other kind of models, robot models outputs.

Vibhu (3:08)

You put out a good blog post around your model. I think a lot of people understand language models or transformers, next token predictors. In this case, you shared quite a bit. There's like diffusion flow matching, you're predicting next world state plus action.

One thing you say is a million hours. How do you determine the architecture? Like what are the choices there? What is the architecture? I feel like in language models, there's a lot of take something off the shelf, take trillions of tokens, do prediction, SFT, RL, and you have a model. What about this? How do you go from nothing to your model? Is there vanilla architecture off the shelf? Do you determine all this? What is the current state?

Jason Ma (3:50)

The mental framework to think about how do we build models for robotics is that there are essentially a couple of things to look at. One is data, and the second is how do we make a model learn as much as possible from that data.

Let's first talk about the model actually.

The blog post we released is our Dyna-2 Word Action Model. Under the hood is a diffusion transformer. Transformer is essentially like the language model, you know, sequence to sequence, match, encoder only. But what's really interesting is that in visual domains, if you think about generating images, generating videos, and in this case, robots are looking at images to generate actions, people have found the diffusion kind of architecture to work really well. So what is a diffusion model? A diffusion model is a model that takes some random noise and through iterative computation can go from noise to something that's really high dimensional, to something really highly defined and have a very fine-grained feature. If you think about generating images, you want sharp edges, sharp corners, color contrast, and for robotics, even though you're not generating an image, you're generating actions that needs to be very precise, very smooth, right? So it has the same kind of consideration of desiderata as generating image. So that's why from first principle, the whole field, not just us, have been using this kind of diffusion transformer architecture to train robotics foundation models. You need the transformer aspect because transformer is the thing that can just ingest the most amount of data, color. Then diffusion helps you to refine the outcome, to get to the smooth, precise, fine-grained motion you need to actually execute on the robot. So that's the architecture that goes behind a lot of our recent breakthroughs.