You cannot tell which parts of your software should stop calling an LLM. My Jev guide has a prompt that scans your projects and names them.

Nate B. Jones (0:00)

Jev is like an LLM that can only talk in multiple choice. It can read whatever you hand it, but it can't write even one word back. So you supply all the possible answers in advance, and it's gonna only pick one. And I know what that sounds like to you. It sounds useless. Why would anybody want a model that cannot write in the age when we have LLMs that are so, so good at writing? Look, TypeSafe shipped Jev on September 15th, and I'll tell you how useful it is, because within 24 hours, it was already the fastest adopted model in Versel's AI Gateway history, faster than any LLM from any lab. You name the lab, faster than that. More than twice as many paid teams have adopted and paid for that product for Jev as for any model ever launched before in the first 24 hours. So either a lot of really, really good developers have suddenly lost their minds at the same time, or the thing that Jev is refusing to do, to write words, is the entire point.

In this video, I'm going to give you four things. I'm going to name this kind of problem so you can spot a Jev-shaped problem in your work. I'm going to tell you the ways people are using Jev in real systems. I'm going to tell you what it costs to try, which is so much less than you think. And I'm going to tell you places where it falls over and isn't perfect because nothing's perfect.

I'm not here to tell you that Jev replaces LLMs, and that's exactly why it matters so much. Let's get into it. Fundamentally, we've been missing a basic building block for the AI age that we're in, and we didn't even know it. We've been missing a general-purpose classifier that's really, really fast, and now that it's here, it looks obvious. And by the way, a classifier, that's just a fancy word for picking multiple choice. So if you hear that, that's what that is. Like, think of it as, I have to pick an answer on my SAT test. We've been missing a general-purpose classifier that's fast enough, cheap enough, and capable enough to put judgment into all kinds of places, where it was previously way too expensive to justify. But to understand what that means, we're going to go back in time. Let's start with something software has always been good at, something that they've been good at for decades, and you'll see where I'm going here. As an example, let's say an invoice is 30 days overdue. You need to flag it. If an order total exceeds a limit, you have a rule to flag that. You get the idea.

These are what we call deterministic rules. The same thing happens every time. You can write the condition down really precisely for that rule, and you can use ordinary code to make that happen. But now let's add a complexity. Let's add something that code doesn't cover. Let's ask, does the customer sound like they're about to leave the contract in this email?

Does this email contain a serious business opportunity, or does it just have the words business opportunity in it? These are things that we as humans really, really care about, and they're things that I've seen AI or LLMs used for today.

But it turns out that they're classifier problems, because what we're really doing is we're reading complicated text and we're outputting an action or a simple choice.

And that turns out to be a huge number of problems in our world. These are problems that we want solved as people, and we've been using LLMs to solve them relatively expensively because they are so valuable. What happens if you can solve them cheaply? Jev gets at what you do when you have text that you want to apply complicated questions to, but you can get a simple set of output options. So you might want a category, you might want a score, you might want a yes or a no to proceed. But to get there, you have to actually deeply understand all of the texts that came before. Stuff that requires interpretation, that requires judgment, right? Customers express dissatisfaction in different ways. If we go back to our customer example, let's say a useful opportunity, it arrives in a badly written e-mail.

Can Jev find that? Certainly, that's challenging to find with rules, and it's somewhat expensive to find with AI, and not always reliable. And so Jev gives us a tool to solve problems that we were solving not at all or expensively before.

This is actually really relevant in the age of AI safety, because right now, agents typically have to ask either humans or LLMs if it's safe to proceed on an action. Jev gives us another tool because it can understand an incredibly complicated piece of agent text and code, and then make a decision, yes, you can proceed, no, you can't, or you should ask a human. And Jev gives us a tool to do that very cheaply, very quickly, which means we can apply it over a much wider part of the agent run. The work I'm describing is what I am calling semi-deterministic. It's a really big word, but I think you're going to start to get it by the end of this video. It's about the glue between language and choice. When we use a tool like Jev, we or an agent can define a short list of available outcomes and what happens after each one. And then we can put an incredibly complicated input into Jev and ask Jev to sort.