Why Local AI Matters and How to Use It

**Nathaniel Whittemore** (0:00)
Today on The AI Daily Brief, how and why to use local AI. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.
All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, Robots and Pencils, Section, Mission Cloud and OutSystems. To get an ad-free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts. To learn more about sponsoring the show, visit aidailybrief.ai/sponsors, or you can email us at sponsors at aidailybrief.ai. To learn more about the new Executive Agent Training Program that Nufar mentioned at the end of this episode, go to training.bsuper.ai. And yes, we are back with another Nufar Operator's Cut. Specifically, this week the conversation has been so much about the changing composition of enterprise AI strategy or just business AI strategy in general, as people deal with one, rising costs from agentic workloads, and two, the new reality that our AI can be turned off on a whim at any moment. And yet there is a chasm between the idea of using alternatives to the major models and actually being able to do so. And so what Nufar has presented today is a primer that's going to give you a background understanding of a lot of the key concepts, terms, and steps you would need to take to even explore thinking in this new way.
All right, Nufar, welcome back to the Daily Brief. How's it going?

**Nufar Gaspar** (1:28)
Good, how are you?

**Nathaniel Whittemore** (1:30)
Good, so we are at the transition point moment. I hope, fingers crossed by the time this airs, although I'm not super optimistic, fingers crossed we might be playing with Fable 5 again. But I think that this week, as I've been discussing all week, has shown why investing only in the biggest model or the best model is not necessarily the best strategy. On Thursday's episode from last week, I talked about what the alternative models and model approaches that companies are starting to think through. But there is a huge gap between just shifting thinking from Fable 5 to some other type of model to understanding what that actually takes.
And that's the gap that you are going to fill in, at least on a basic or high level for us today.

**Nufar Gaspar** (2:16)
I'll do my best. All right. So I do think that there is a big gap between saying open source and fully understanding the implications and deciding whether you should go and buy a hardware for your company. There is a lot of understanding that needs to be done of what it all means. So today I'll try to give a very practical overview of why you should care about open source and why you should care about running models locally.
In practice, it will also include how can you do it and for whom it might be relevant. So just a quick recap of the perfect storm that makes open source so important nowadays, in my own words. So the first force that I see is the cost axis. Everybody is talking about tokens, the cost, and how to maximize value while minimizing the cost or optimization of the cost.
Anyway, a lot of conversation around tokens. That's for a good reason, but for the most part, it's becoming more and more expensive. Just a few examples that I'm sure many of you have encountered, the price growth of GPT, the release of Opus 4.7, that changed the tokenizer, and all of a sudden companies who hasn't made any change to their prompts has a bill that increased in sometimes 35 percent. And we're seeing more and more companies leaning towards agentic workflows and as such, these harnesses are also a huge cost multipliers. So the thing is that many individuals and companies are more than willing to pay the price if the return is justified. And we're all very excited and pleased to know that there is a new player in the block and namely Fable 5 And then as we all know, it was shut down and we're still kind of waiting to understand the events and maybe by the time it ends, your optimism is going to be in motion and we're going to have Fable 5 back. But I think that the AI is becoming increasingly more volatile because of the geopolitical forces. So theoretically, we said it before, but I think that seeing that in action and realizing that all of a sudden you might have a high dependency on a single vendor that can be shut down by a government that create a new category of dependency risk that we need to start thinking about how to alleviate that.