Muse Wonder: Zuckerberg and Wang's $100B AI

Mark Zuckerberg (0:00)

At the end of last year, we're like, all right, we are quite excited about the progress. We haven't seen the models yet.

We're going to write this check for like, I don't know, like $100 billion or $200 billion. I'm like, Alex, you sure this is going to work?

John Collison (0:16)

Cheers.

Alexandr Wang (0:17)

Cheers.

Mark Zuckerberg (0:17)

Yeah.

Alexandr Wang (0:18)

Cheers.

John Collison (0:19)

Cheers.

Mark Zuckerberg (0:20)

Let's launch it.

John Collison (0:24)

So, give me the update on Muse.

Mark Zuckerberg (0:27)

I think if you were looking at the third-party reports, you would see that this might be the fastest growing agent or AI app that I think has existed yet.

John Collison (0:41)

What are your guys' personal favorite Muse use cases? Either yours or other people's?

Alexandr Wang (0:47)

I was actually talking about this at dinner yesterday. My favorite thing is I have my Muse, whose name is Euler, constantly source increasingly obscure teas from increasingly obscure vendors of tea. So at this point, he's on a weekly basis, my Muse is buying some random tea from some farm in Taiwan or some farm in Japan and sourcing it over and bringing it back.

John Collison (1:16)

What's your favorite Muse use case?

Mark Zuckerberg (1:19)

I don't know. I use it for a ton of stuff. I still like, my favorite both use case and probably Eval for the product is having it coach me in MMA. So I put cameras in my gym and I basically, it watches the cameras when I'm in there, and it gives me feedback in between the rounds, which is a very hard problem, it turns out because not only does it have to, but it basically has to process all the video quickly enough because you only get one minute in between the rounds.

John Collison (1:52)

Oh, it coach you between the rounds? Yeah. I want feedback. I want feedback on how to win. Did you have to instruct us? I've wondered about this too because looking at videos is a really good AI use case.

It feels like you can't just give it a multi-gigabyte video file. So do you tell us how to slice stills out of it? No.

Mark Zuckerberg (2:10)

Basically, yeah, it pulls stills. I think it figured that out by itself.

It turns out that Muse has pretty good native understanding of MMA. So that's good. So the multimodal- It's in the model.

Alexandr Wang (2:25)

Yeah.

Mark Zuckerberg (2:27)

Someone on the team knew. Now we're just out there in the corpus.

I think it's a good test of the coding ability. Because really, basically, what it's doing, it's like within the agent, and it's like, I'm not seeing any code, but it basically is writing this whole video processing image, processing pipeline, and I'm constantly telling it like, all right, you look like you took 30 seconds to give me that feedback. I want to have 60 second break. Like you have to go faster. Just optimize yourself. Optimize yourself. So it's, I think, a good example of how stuff is going to go in the future, where basically everyone is actually going to be writing tons of code. They're just not going to know that they're writing code.

John Collison (3:08)

What kind of MMA steers does a model give you?

Mark Zuckerberg (3:13)

You want to see?

John Collison (3:14)

Yeah.

Mark Zuckerberg (3:14)

Basically, let me see if I can pull some stuff up. But it's basically, it gives me a lot of stuff about balance and hip position, or it's like, when you're inside control here, it's just like you didn't get this underhook in this place, so you're vulnerable to this getting rolled over. Here, I'll pull up some stuff.

Alexandr Wang (3:35)

It was definitely one of his first use cases, actually, of even the initial prototypes.

Mark Zuckerberg (3:39)

Yeah. I did this with OpenClaw before we even had Muse working.

John Collison (3:43)

Yeah.

Mark Zuckerberg (3:43)

It was just like, because then that's what I was saying, it's become an eval for this too.

Alexandr Wang (3:50)

I actually didn't know that it was an eval for the product, so noted.

John Collison (3:54)

Yeah.

How is all this in the models training data, like the fine-grained, this is what an MMA fighter looks like when they're appropriately balanced, and this is when they're like too much on the back foot. I'm just wondering, how is it consumed so many images of appropriate versus inappropriate MMA posture?

Mark Zuckerberg (4:17)

Here you can see, it just pulls out frames from it, and it's like, okay, I'm like too much on my back leg there.