Fugu vs Fusion: The Fable 5 Test

**Julian Goldie** (0:00)
Fugu vs Fusion, who wins? So we have two different APIs that just got dropped, and both of them are claiming that they're beating Fable 5 on the benchmarks, and I'm actually testing them out side by side. If you don't understand what these APIs are, basically, Sakana just dropped, it's a new model called Fugu and Fugu Ultra, and that is from Japan. And then we have another model called OpenRouter Fusion from OpenRouter, obviously, and that is an API you can use in the same way.
And what happens is essentially you do one prompt, and with Fusion, you have five models or a few models of work together.
They will come up with an answer, the judge critiques and analyzes them, and then you get one single answer. Similar to Sakana, where you ask one thing, you get multiple models to answer, and then that gets synthesized into this setup here, and you get one answer. So it's like a panel engine. And these panel APIs both claim to have better performance on benchmarks than Fable 5 Now, obviously, you can't access Fable 5 It's been taken down. It's gone. But you can access OpenRouter Fusion on API, and you can get access to Sakana. So how do they perform on the benchmarks? Let's take a look. So I've been testing out, and bear in mind that it's quite limited with Fugu, and also it takes a long time to get a panel of AIs to give you an answer. But if we compare these side by side, we've got Fugu Ultra, we've got Fusion, and we have Fugu Mini, which is the other option. So let's open up, for example, these three websites all creating the same landing page. And this is Fugu Ultra. Pretty nice. Looks super nice. I mean, the design is really, really good there. Nice UI, et cetera. If you compare that to Fusion, we've got Fusion's. I'd say it's still pretty nice, but I wouldn't say it's quite as nicely designed. And then we have Fugu Mini, which is, it looks pretty cool to be fair. I'd still say it's probably better than the design from OpenRouter and Fusion. So overall, I would say Fugu wins. Fugu is kind of going to win anyway, because it gets two votes, right? Because it's got two models, Fugu Ultra and Fugu Mini. Fusion just has one. Now let's check the next test, which is an Orbit Galaxy test. And there are pros and cons to both, which I'll come on to in a second as well. So we've got the Raycaster Maze test over here. This is Fugu Ultra. Looks pretty nice, pretty cool to use.
Super smooth when you use it as well. It looks really cool. And then if we test it out versus we've got Fusion over here, I would say both good outputs. Marginally, I would say that Fugu wins on that test as well. So Fugu is winning on both tests so far.
Both the Raycaster and the landing page. Now we have a galaxy simulation here. So this is the living spiral galaxy test, where you can drag the camera, you can zoom in, you can zoom out. And look how beautiful that looks, by the way. Like how insane is that? It looks absolutely amazing.
You can also double click to recenter, zoom in, zoom out. But it does, that's probably the best living spiral galaxy simulation that I've ever seen, to be honest, from AIs. It's unreal. Now let's compare that to Fusion, which is over here. It's kind of, it just doesn't feel quite as smooth, if that makes sense. Like look at that, and the quality of that, compared with the quality of this. This is way more beautiful, way nicer. So Fugu Ultra is winning on three different tests right there. Now let's have a look at the inner solar system test as well. So this is Fugu Ultra and then we have OpenRouter here as well. And this is again, like a big difference in quality, like Fugu's version here just looks so much nicer. It's so obvious that it's nicer than this version right here. So it's absolutely crushing Fusion on tests, is beating Fable 5 on benchmarks, but so is Fusion. There are some downsides to this. So I mean, number one, you'll see, for example, we haven't generated all of the tests that we normally do. And the reason for that is because it was actually token limited. Even when I upgraded the plan for Sakana, I mean, it's only just dropped today. It's only just dropped a few hours ago. So I will test it more once it's available to test more on the API plan. But just one thing to note here is like Fugu is pretty limited in the amount of generations you can do, both by the time that it takes to get generation and then also by how many generations you could do per day. So I hit like a five hour limit pretty quickly and then obviously I have to wait five hours to use it again. With Fusion, that's not so much an issue because as long as you've got the API, which you can get from OpenRouter, you can then plug it into something like this. We've actually created kind of like a nice looking chat box where we can ask Fusion any questions and then everything we build with this goes directly into our workspace right here. And we actually did the same with Sakana and its API as well. But again, Sakana is a little bit limited. I'm sure they'll improve that in the future because it's just dropped today.