Ep 865: Open Source AI 101: Why Local Models, Cheap APIs, and AI Agents Change Everything (Start Here Series Vol 24)

Jordan Wilson (0:00)

Welcome to the Everyday AI Podcast. My name is Jordan Wilson, and for the past three and a half years, we've put out more than 800 episodes. Yet, one of the most common questions I get, I didn't really have an answer for. Where do I start on the Everyday AI Podcast? And that's why we started the Start Here series. And with the fall now back in full swing, the Everyday AI Podcast is going back to school and playing back the entire Start Here series from front to back. We've hit pause on our normal Monday to Friday programming to run back our most popular series ever for the next 30 days. We made the Start Here series for beginners and AI champions alike. So whether you're just trying to get a grasp on large language models or grappling with the best coding harness for multi-agentic workflows, the Start Here series covers it all. Plain language, no jargon, and easy to follow along each day. So make sure to subscribe to the podcast and check back each day for new insights day by day. The series is a culmination of spending more than 10,000 hours covering generative AI over the past three and a half years. So you don't want to miss a single episode of the Start Here series. Let's get into it.

A few weeks ago, the United States government said the quiet part out loud when it comes to open source models, at least from China. That's because in April, the White House sent out an official memo accusing China of using distillation to illegally copy American AI models to create cheaper domestic knockoffs. And that declaration is really nothing new if you've followed AI for years. However, the recent distillation trend has completely reshaped one important landscape of enterprise AI, the decision between using open source models versus proprietary closed models. And in 2026, at least, it can actually be a tough choice between saving potentially millions of dollars versus running up your legal liability. About two years ago, before Chinese distillation was commonplace, there was a sizable gap between frontier models and open source AI models, or those models that you can essentially download or use for close to free. But now, the gap is all but closed, which has thrust the open source versus closed source question into every enterprise boardroom in 2026 And although there's no one size fits all answer, we're going to be tackling the toughest topics and the most important takeaways as we take a zoomed out view of open source models on today's show. That's why we're going over open source AI 101, why local models, cheap APIs and AI agents change everything about making AI decisions in 2026 as part of our Start Here series. All right. Welcome to Everyday AI. Before we dig in, let's first zoom out and talk about the big picture here when it comes to open source AI. That's because, well, it's actually a legitimate thing now.

Two years ago, enterprise companies weren't saying, let's use an open source AI model in production. Today, it's actually happened. It's happening. That's because maybe the most powerful open source models are only about two to six months behind frontier models. But on even consumer hardware, you can be running essentially frontier level AI models from like just over 12 months ago. And the Chinese labs now distilling US models have kind of crashed the open API prices to pennies, right? So yeah, not everyone out there on, you know, consumer or prosumer hardware can run the most powerful open source models. Although I do think Google has something to say about that, but the most powerful open source models run for a fraction of the actual cost if you can't afford to run them locally, which has completely shifted the paradigm when it comes to enterprises making decisions on, well, are we going to use a model from one of the big three open AI, Google or Anthropic, or are we going to use a Chinese open source model and pay for it that way? And well, what this has also led to in 2026 is essentially 24, seven local agents that can run and also without costing a ton. And now having to actively and almost aggressively, aggressively go through kind of an AI cost triage. But going full open source does strip away the legal protection that the closed models often include.

So stick with me for 25-ish minutes on today's Start Here series show. And here's what you're going to learn. You're going to know why the open versus closed source AI default just officially flipped. You're going to know how Gemma 4 from Google puts year-old frontier capability on your laptop. You're going to understand the two payoffs already shaping and reshaping how individuals and enterprises run AI and the hidden legal trade-off most executives miss when going fully open source. Let's get into it. My name is Jordan Wilson. Welcome to Everyday AI's Start Here series. This is the essential podcast series to learn the AI basics, and if you're an AI expert, this is your chance to freshen up and double down on your AI knowledge. Why do we start this Start Here series? Well, after 750 plus podcasts, I never really had a good answer when someone was like, where do I start? What podcast do I start with? That's why we created the Start Here series. It's best, I think, if you listen in order. I think this is now volume 24 of the Start Here series, so maybe we'll wrap it up at 25, maybe we'll wrap it up at 30 I'm not sure, but the whole point of this is you can go to starthereseries.com. That's gonna give you free access to our exclusive Inner Circle community, and in the Start Here series space, we make it even easier for you. So you can actually go listen. We have a Spotify playlist ready for all of the different Start Here series shows, as well as a breakdown on each individual episode, all in one place, so make sure you go to starthereseries.com for exclusive access to that inside of our inner circle community. All right. And if you missed our last Start Here series show, that was Volume 23 We talked about headless software and why companies are building software for AI agents and not humans and well, what that means. So today in Volume 24 of the Start Here series, we're going over open source AI 101 So here's the reality. Closed AI used to be the de facto, right? And I mean, honestly, there was really never even much of a discussion about open source AI in the enterprise, maybe until 2025, at least not serious enterprise companies. Now it's a real conversation, right? So it's no longer, you know, hey, we're just going to choose whichever API works best for us, right? Whether that's Open AI, Anthropic or Google. Now, most companies are looking at some of the open source alternatives, most of them coming from China. And the big kind of thing here is companies are starting to standardize around one frontier vendor in 2025 before this happened. And then they called it an AI strategy. And the assumption originally was, well, that worked for three years until two very specific forces broke the standard paradigm when it came to open source models.