Jev + Needle 3 AI Started Something NEW!

Julian Goldie (0:00)

Needle 3 might be the biggest shift in AI agents we've ever seen, and it just dropped. It's a tiny new AI model from Cactus that runs on your phone, your laptop, even a watch with no internet at all. And here's the part that sounds made up. It beats AI models ten times its size. Completely free, it's open source, and it answers faster than you can blink. In this video, I want to show you what Needle 3 can do, how it teams up with another new model called Jev to give your AI agents a serious upgrade. And the one thing you absolutely have to get right, or your results will fall apart. I'll show you that near the end, so stick with me. By the end of this video, you'll understand the shift that's about to make AI agents faster, more reliable, and free to run. Let's get into it. So Needle 3 runs in 66 milliseconds on a normal CPU with no graphics card and no internet, and the whole model is 36 megabytes. That's more than a lot of the apps sitting on your phone right now. So let me start with what actually happened in a live test, because the numbers, the whole reason to care. Someone spoke a messy sentence out loud, something, for example, like, I'm feeling really hot today. Can you turn on the fan? Turn the temperature down to 10 degrees and turn on the bedroom light. Three separate things said the way a normal person talks, not like a clean command, not a list. The model turned that into exact actions.

Fan on, temperature set to 10, bedroom light on, correct value 66 milliseconds on a CPU, not a server farm. And Cactus, the team behind it, says two function calls come back in under 97 milliseconds on a CPU with no graphics card at all. Now, why does that matter to you if you're not building smart home gadget? Stay with me because the part on this that you need to know changes how you think about AI agents. And it has nothing to do with light switches. So first, let's talk about the size. Needle 3 is a single file between 8 and 29 megabytes, depending on how you slice it. The full version people are testing sits around 35 megabytes with the engine included. If you compare that to a normal small AI model, the 7 billion parameter model squeezed down to 4 bit is still about 4 gigabytes. Needle 3 is smaller than a lot of speech-to-text models. It's small enough to sit inside a watch, and it's Apache 2 licensed, it's free, it's open source, and it's already sitting at over 11,000 stars on GitHub. Here's what it actually does, and there are three things. Number one is tool calls. You can show it the functions your app has. It picks the right ones and fills in every blank from what the person said. So if you ask for two things, you get two calls in the right order. If you ask for something none of the tools cover, it gives you an empty list instead of making something up. Number two is structured extraction.

So you can declare a shape, say, vendor, total, due date. You hand it messy text, and you get clean fields back. That could be an invoice, a booking, a form. And the way it writes the output means the result always passes. It can't hand you broken data. And number three is embeddings. So the same file turns a sentence into numbers, so an app can search and match things on the device without sending anything anywhere. One file, three jobs, all offline. Now, practice were honest about the trade. They gave up general chat ability on purpose. If you ask Needle 3, the capital of France, and you get nothing back, it's because there's no tool for that. It's not broken, that's literally the design. And that trade is why 121 million parameter model beats models 10 times its size on mobile tool calls, and matches models two to three times bigger on extraction. Now, let me run you through the actual benchmark numbers because Cactus have published them all. They tested on six suites. Mobile actions, 961 rows of phone commands turned into Android actions scored on exact MACDroid call.

200 phone requests, some leading to calls, and that was scored on getting the calls exactly right and in order.

BFCLV4, and that's the Berkley function calling benchmark. 3,641 rows, and also checks whether the model correctly refuses when nothing fits. And then there's three extraction tests, which are DSTC8 with 1,813 dialogue turns, SNIPS Gold with 700 rows where you hand out the schema, and SNIPS7 Waze with 700 rows where it has to pick the right schema out of 7 first and fill it. And these aren't like small samples, and the comparison models were run full precision under VLLM, whilst Needle 3 was run through its shipped 2-bit compressed file. So the tiny one was handicapped and still won its category.