Open Models Change The Economics of AI

Jeffrey Morgan (0:00)

Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their north star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case.

SPEAKER_2 (0:20)

Early 2024, there was lots of interest in fine-tuning your own custom models. Then it sort of went away, or there would just be wasted effort, or get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Do you think we're going through another cycle, or is it here to stay this time?

SPEAKER_3 (0:43)

Welcome back to another episode of The Lightcone. Today we're talking to Jeffrey Morgan, co-founder and CEO of Ollama, the easiest way to run open-source AI models locally and in the Cloud. Ollama is used by 9 million developers, has 178,000 GitHub stars, and is used by 85% of the Fortune 500, which means Jeff knows a lot about the state of the art of AI, what model score highest on benchmarks, and what developers actually download and keep using. Jeff, welcome to The Lightcone.

Jeffrey Morgan (1:16)

Thank you for having me.

SPEAKER_3 (1:17)

We're down to here. What is the state of the art? What are you seeing out there?

Jeffrey Morgan (1:20)

Well, I think the biggest thing we're seeing is a shift to open models, especially in enterprise.

And that's from a mix of US and Chinese origin models. And it's predominantly driven by coding agents, and also AI assistants, more co-work use cases, like OpenClaw and Hermes.

SPEAKER_4 (1:36)

And because you sit in the token flow of like so many tokens, you have really good data on what models people are actually using and how it's changing. What are the trends that you're seeing?

Jeffrey Morgan (1:45)

Yeah, you know, Ollama started as a way to run open models on your MacBook or other hardware, Nvidia, AMD, Intel. And earlier this year, we launched Ollama's Cloud. And what we're seeing there is that it's predominantly Chinese models right now of Chinese origin, but they're being accessed by businesses all over the world.

Especially US and Germany is actually a big source of where open model tokens are being accessed.

SPEAKER_2 (2:13)

Is it all about cost? Is it our enterprise is coming because they just want to get the cost down or is there anything more to it?

Jeffrey Morgan (2:18)

Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case.

SPEAKER_5 (2:39)

Is there a particular large enterprise that you can name that has done this?

Jeffrey Morgan (2:44)

I think there was a great article in the information yesterday from AT&T, and it ends up they've already shifted 40% of their token consumption to open models. And that's right now predominantly through US and Europe models, but they're also evaluating the Chinese models.

SPEAKER_5 (3:01)

What kind of workflows do they run?

Jeffrey Morgan (3:03)

Predominantly coding agents. I think what we've seen just from the extreme growth and per developer or per user token usage has predominantly been from coding agents. And then earlier in March and April, we saw OpenClaw take off and subsequently the Hermes Project, the Hermes Agent Project take off, which has then opened up that ability to automate a huge chunk of work over a long span of time to non-developers too, whether it's like finance or support or marketing or sales.

SPEAKER_5 (3:31)

You had this actually very cool graph on the takeoff exponential for OpenClaw.

Jeffrey Morgan (3:37)

Yeah. So earlier this year, this is a graph of token usage by developer on Ollama's Cloud, the average amount of tokens they use per week.

SPEAKER_4 (3:48)

This is per developer. So this graph looks like it should be an aggregate of Ollama's growth, but this is actually the per user. Exactly.

Jeffrey Morgan (3:56)

This is on an individual user basis. How many tokens are they using a week?

There's two big inflection points. One is that initial run up at the start of the year, which was driven by coding agents. So we saw Kimi, the GLM models, Minimax, Launch. Finally, we had Open Models that could power coding agents. And then in April, we saw this incredible growth from OpenClaw, really, which was then not just developers, but the rest of the world could take a hard problem, give it to an Open Model and let it go complete the task, which obviously consumes a ton of tokens as it's figuring out what tools to use, what data to go fetch. We went from a context window of 128k to a million plus with Open Models. And so all that enabled this explosive growth.