The Right Model. The Right Place. The Right Cost.

Darren Pulsipher (0:00)

Welcome to Embracing Digital Transformation. Before we dive in, I wanted to personally thank you for listening. Many of the ideas we discuss on this show inspired my new book, AI Augmented Teams. If you're looking for practical ways to combine human expertise and AI to achieve better outcomes, I think you'll find it valuable. Learn more at paydar.ai.books.

That is paidar.ai.

Now, let's get started with the show. Welcome to Digital Transformation Architect. I'm Dr. Darren. Today, we're exploring the right.

We'll look at what it takes to turn digital change into lasting results. Let's get started. Welcome. Today we are going to talk about a very practical question.

How should AI actually run? This is not a model shopping exercise. We are going to start with real work, connect that work to an outcome, and then decide what model, what location, what controls, and what cost makes sense.

So let me start with the question. Where should AI run? On the user's device? At the edge? In your data center? In the cloud? Or across several of them? Most teams answer cloud before they have described the work. Let's slow that down. The answer depends on the workload.

Think about these four Clerk of Court workloads. Summarize a public document. Search internal procedures. Assist with sensitive case information. Automate a multi-step workflow. Where would you run each one? Would you make the same choice for all four? Probably not. They use different data, have different consequences, and need different levels of latency and control.

For a long time, the architecture question was, which model should we use? That question is still there, but it is no longer enough. Now we also have to ask, which data will the model use? What controls are required? And where should this work run? If we cannot answer those questions, we do not really have an architecture yet.

This is the framework for today. The right model, the right place, and the right cost. Those three decisions should lead to the right outcome. And notice what we are optimizing. We are not just trying to find the cheapest token. We are trying to achieve a trustworthy result at a cost the organization can actually sustain.

When we finish, you should have one real workload defined in business terms. You should have a first placement decision. You should know which architecture pattern fits, what controls and grounding it needs, and what barrier is keeping it from deployment. And you will have a canvas you can use again. So please work on something real.

The first mistake is starting with we want to use generative AI. That is not an outcome. Start with the work. For example, we want to reduce the time employees spend finding an authoritative procedure from 20 minutes to under 3 minutes. Now we have a user, a task, a baseline, and a target. That gives the architecture something useful to serve.

Let's frame the workload in five parts. User, Task, Current Process, Desired Outcome, and Measure. Who is doing the work? What are they trying to do? What happens today? What should improve? And how will we know?

Take a moment and write one sentence for your own workload. If the outcome is vague, every technology decision that follows will be vague too.

Now look at the compute continuum. Device, Edge, Data Center, and Cloud. This is not a ladder. Cloud is not more advanced than a device. A local model is not automatically better than a hosted model. Each location gives us a different mix of control, latency, capability, scale, connectivity, and cost. We place the workload where those requirements fit.

So what drives placement? Start with six questions. Where can the data go? What information can leave the environment? How fast does the answer need to be? How capable does the model need to be? How much volume must we support? And what does one useful outcome cost? Those are business questions, but they directly shape the architecture.

This is where bigger is not always better. A large model may give us broader capability. A smaller model may be faster, cheaper, more private, and easier to constrain. A specialized model may be the best choice for a bounded task. We may even use several models and route work between them. The question is not, what is the biggest model we can use. The question is, what capability does this workload require?

I like to think of an LLM as a probability engine. Given a prompt and some context, it predicts what should come next. Different models give us different distributions of likely answers. A large model may cover more kinds of work. A smaller or specialized model may be much tighter for one task. So the interesting question is, what can we do to shape that distribution?