Meta Just Dropped Its Most Powerful Open-Weight AI!

Julian Goldie (0:00)

So, Meta have just released Muse Spark Open. They call it the most capable US open weight model to date. And it runs on two Apple Mac Studios sitting next to each other, so you don't need like a big data center or a cloud. You can just have two boxes on a desk. Mark Zuckerberg actually said it runs great on the new Mac Studios, thanks to work with John Turnus and Apple. Turnus actually runs hardware at Apple, so this came out of Meta and Apple working together. Now, why should you care if you're not a tech person? Because until now, the smartest AI used was always rented, right? It lived on someone else's computer. You typed in your stuff, it went off to their building, and it came back. Musespark opened as the first model, this strong. The potentially, if you had two Mac Studios working together, you could actually download it, keep it running at home, right? And your files would stay with you, and your agents keep working even if a company changes its rules tomorrow. So this is the announcement, and you can see here for the first time, it's running on Apple 5 Ultra Mac Studios. And let me give you the numbers first of all. It's 816 billion parameters, there's a 1 million token memory, and two Mac Studios to run the whole thing. Now, what's the problem we're solving here? Well, let me define one word first, which is weights. Weights are the brain of an AI model. It's one giant file, right? And everything the model learned is packed inside it. When a company says a model is open weight, it means they let you download that brain and keep it. Now, most top models don't let you do that. You can use them, you can pay for them, but you never actually get the original brain behind it and the data behind it. With this system, with MuSpark open, this is basically Meta's answers to that. And you might be wondering, okay, what's actually different here? So three things. First one is the size and how it uses that size. So 816 billion parameters, but only 42 billion are working at any one time. And it's a mixture of experts model. So let's say, for example, I don't know, you have a kitchen with 816 doctors, sorry, a kitchen with 816 chefs, right? And then you have 42 chefs that actually wake up and work when you ask it to do something. That's basically how this model works. And the second thing is the memory. So it's 1 million tokens, 1 million token context window, as you can see right here. Now a token is a small piece of word, 1 million tokens, roughly about 750,000 words. So that's around 10 long books at once. And the third thing is that it's natively multimodal. That means it was basically built from day one to understand text, images, video and audio. Now they've actually tested it on Muse code, as you can see right here. So it's optimized for Apple Silicon. And they were using the latest version of MLX. It was optimized to run on Mac Studio with M5 Ultra.

And they've actually said together, they're the best way to run a powerful model on your desktop. So Apple's unified memory architecture lets the M5 Ultra's integrated GPU use up to 512 gigabyte per Mac Studio. So the way this works is like you would have two machines that hold the entire model. And then it's got a chip of 1.2 terabytes of memory bandwidth, which is fast enough to read the small slice each token needs. And so like when you have two Mac Studios linked together on the M5, basically they would just work like a supercomputer together, which is pretty crazy. Now you might wonder, OK, how does that work? Well, basically the new M5 Ultra chip uses something Apple calls unified memory. Normally a computer has two kinds of memory, right? One for the main chip and one for graphics. They have to pass stuff back and forth. But with unified memory, it's basically one big pool that everyone shares. And each Mac Studio can give up to 512 gigabytes of that pool to the graphic side. Now two of them together, that's about a terabyte, and that's enough to hold the entire model. Then there's also the speed. So the chip moves memory at 1.2 terabytes per second. That's fast enough to grab the smallest lives of the brain. Each word needs. A meta says the two machines basically act like one supercomputer together. That's a big shift, because like a year ago, a model this size would mean like you need massive servers.