A Practical Guide to Scaling AI

**Nathaniel Whittemore** (0:00)
Today on The AI Daily Brief, a practical guide to scaling AI. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.
All right, friends, quick notes before we dive in. First of all, thank you to today's sponsors, KPMG, Robots and Pencils, Blitzi and Robo. To get an ad-free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts. Remember, for those of you who hate ads, ad-free is just three bucks a month. And if you are interested on the other end of the spectrum in sponsoring the show, shoot us a note at sponsors at aidailybrief.ai. Welcome back to the AI Daily Brief. Today, we are talking about some practical, useful frameworks for scaling AI. In other words, moving beyond the pilot stage. And the specific frame of reference and framework that we're going to be using comes from a new guide from OpenAI called From Experiments to Deployments, A Practical Path to Scaling AI. But even outside of this particular document, it is very clear to me that a huge theme heading into next year is going to be this idea of whole org transformation and post-pilot, post-experimentation phase artificial intelligence inside the enterprise. If you look at literally any study of AI adoption and impact, the story is pretty clear. Massive and increasing adoption and usage, initial extremely promising ROI and impact, but some real barriers to converting individual value to whole org value. Indeed, the very first set of charts in McKinsey's State of AI doc shows this really crisply. On the one hand, the percentage of organizations that are using AI in at least one function continues to reach new highs, and increasingly it is spread across multiple functions. But a huge number of organizations remain in really early stages. McKinsey found 32% still experimenting, and another 30% in the piloting stages, meaning just 38% total were scaling or fully scaled, and only 7% of those were in that fully scaled phase. Now, to be honest, the 7% in fully scaled, I don't think is all that bad. When you have a technology that is going to impact every single part of the organization, I would actually be surprised if those 7% actually are fully scaled. There's just so much to do to get there. The more concerning piece, especially if you are in that cohort, is the 62% that are still in those really early stages. I genuinely think that if you head into 2026 in those stages, you have to treat yourself as officially behind. One thing we've recently done at SuperIntelligent is as part of our Agent Readiness Audits, we were very frequently recommending some version of Quick Win pilots as part of the initial things to do, especially for organizations that found themselves in the Explorer stage, which is very early in their AI journey. I've now basically hard blocked and demanded the removal of any sort of mention of Quick Win pilots. I just think that if we're talking in that sort of language, and acting like a couple of Quick Wins is an okay place to be, it's doing our customers a disservice. This does not mean that I think that organizations have to have everything wired right now. I think it's fine to build momentum with pilots that show value quickly. But I think that the frame of reference and the overall vision for this has to be systemic. I think organizations need to be thinking comprehensively and systematically, or else they risk falling farther and farther behind. Which brings us to this new guide from OpenAI. From experiments to deployments, a practical path to scaling AI. Now, the folks over at OpenAI have been producing this sort of resource more regularly. And I think what's valuable about this is not just that it's a sort of from the horse's mouth kind of document, although that is useful. It's also valuable because it reflects not just their best insights, but the aggregated wisdom that comes from their boatload of enterprise relationships. Kicking us off, they reiterate this problem that we've been talking about for the last couple of minutes. That there is increasingly a divide between laggers and leaders, and while some get stuck in pilots, others are weaving AI into daily operations and customer products. Now, they don't put it quite this crisply, but to me at Core, there are four big mental shifts that OpenAI is suggesting as a basis for all of this work. The first is a shift from thinking about tools to thinking about systems. In their introduction section, they write, For years, companies have focused on validating whether software was fit for purpose. The approach was simple. Start small, test a specific use case, and scale once results are proven. This worked when technology evolved slowly and served a single department at a time. AI moves differently. Its capabilities evolve in weeks, not quarters, and its impact reaches every part of the organization. Success depends less on a single tool's performance and more on how quickly teams can learn, adapt and apply AI to solve the problems in front of them. These shifts demand a new operating rhythm that balances speed with structure and evolves as fast as the technology itself. I actually think that breaking out of the tools based view of software is way more challenging than many organizations are realizing. When you think about the entire structure of information around technology and innovation, so much of it is anchored to this old tool based world. The biggest enterprise research and innovation company in the world is Gardner. $6 billion a year in revenue, $20 billion in market cap. And their most popular tool is their Magic Quadrant. The Magic Quadrant is of course all about helping enterprises pick which tools. They divide things into a quadrant of categories like challenges, leaders, niche players and visionaries and plot companies on that access. The problem with this of course is that when it comes to AI, the difference in your organizational success will have almost nothing to do with whether you choose OpenAI or Microsoft Co-Pilot or Google Gemini, sorry to all my friends who work at those companies. That's not to say that different models won't be better or worse at different purposes. But when it comes to whether an enterprise gets the most out of AI, it will absolutely be based on how good are the systems that they put around AI. And so this entire tool-based frame of reference kind of needs to get booted out the window. So that's the first big mental shift. The second big mental shift is just thinking at a new velocity. Part of the reason that we can't get stuck in the tool-based way of thinking is that the tools themselves don't stay in one place for very long. One of the more remarkable charts in this entire document, they went back and looked across ChatGPT and the API and found that there had been a new feature released approximately every three days this year. That is absolutely insane. And it also creates an incredible organizational burden for companies that are trying to adopt all of that new capacity. It has been clear for some time that the capability set of AI tools vastly outstrips the ability of business users to put them into practice. And I don't see that gap doing anything but expanding. A third big mental shift has to do with leadership and innovation. And there are really two parts of this. The first is that because AI is crosscutting, innovation that happens in one team can actually be relevant for another team in a way that was not the case before. When your sales team was using specialized data enrichment software to help with its sales prospecting, that wasn't necessarily going to help marketing. However, now, there are certain types of prompts and use cases that sales could discover that would be useful for marketing. As OpenAI puts it, innovation can come from any team. A marketing analyst, they write, who automates reporting can find use cases that scale across the whole company. And that gets at the second part of this shift. Solutions from anywhere doesn't just mean from any team, it also means from any type of employee. There is no seniority level that is a prerequisite for figuring out how to use AI better. In fact, one of the things that we talk about very frequently on this show is how that it's still early enough that there's basically no experts, just people who have more time on task and more reps with these tools. All that comes together for OpenAI to present a vision of compounding ROI. And I think this is really valuable. It's very easy to get stuck in thinking about different types of impact or different types of ROI as disconnected from one another. In other words, this AI use case is a time saver, this use case is a cost saver. That really exciting one, that's a new revenue generator. OpenAI is suggesting that instead we think about these things as cumulative and linked and ultimately compounding. Okay, so four big mental shifts from tools to systems, speed of change, solutions from anywhere and compounding ROI. So what overall is the AI framework for creating a repeatable system for scaling AI? Four parts. The first is setting the foundations, establishing executive alignment, governance and data access. The second is creating AI fluency, building literacy, champion networks and sharing learnings across teams. The third is scoping and prioritization, capturing and prioritizing ideas through a repeatable intake process focused on business impact. Fourth and finally, building and scaling products, combining orchestration, measurement and feedback loops to deliver safely and efficiently. You can see they put it in this cumulative and repeating cycle where from those foundations, you layer AI fluency, scope and prioritization and then building and scaling products and have that iteration cycle throughout. So let's talk about foundations first. Within each of these categories, OpenAI gives a set of steps, almost like a recipe for what an organization could do to start to think in these more systematic terms. So for example, in the context of the foundation step, step one they have is assessing your maturity, step two is bringing executives into AI early, step three is strengthening access to data, step four is designing governance for motion, step five is setting clear goals and incentives. Now a lot of our work at SuperIntelligent is of course in and around these zero to one moments, and so a lot of it is resonant. That maturity assessment is an incredibly important step because usually what you're going to find is that an organization's readiness for AI and agents is very jagged. There are certain pockets of the organization that are ahead and optimized for exactly the sort of iterative adoption that is required, whereas other parts of the organization, not necessarily the ones that you would think, might lag for reasons that aren't just technical. But the point, of course, is that you have to know where you stand before you can build a program to move the whole organization together. On the idea of bringing executives into AI early, it is absolutely true, but one important note that builds on what we've seen is that this needs to be a two-way buy-in recruitment. Yes, executives need to be brought into AI early and be seen to be using these tools and changing how they work because of them, but they also need to have a ground level view and a pulse from what employees are thinking. I think that when ChatGPT first came out and enterprise adoption first started, people might have assumed that it would be executive buy-in that was the blocker, but in fact, it has often been the opposite, where executives get super excited and actually kind of exhaust their employees by pouring too many new things on them at once. Both are important, you just have to have a bi-directional conversation about buy-in from all different parts of the organization. On the idea of governance, you might remember that when I did some research and analysis across the thousands and thousands of interviews we've conducted, one really interesting stat that stood out was that organizations that had robust articulated governance programs around AI scored on average 6.6 points higher on our 100-point agent readiness scale. It was the single biggest differentiating factor in terms of how much impact it had if that governance program was there versus if it wasn't. They suggest creating a cross-functional center of excellence, and that's a pattern we see a lot. A last note from Foundations is around the data. They write, reliable data and tools underpin every AI initiative. Start with low sensitivity data sets to move quickly while improving quality and governance in parallel. And again, this is just a framework, but this, I think, actually reveals how much easier it is to say this stuff versus actually do it. In fact, when I was thinking about the Foundations piece, I think it might be valuable to think about Foundations not as the thing that you do in day zero before the other phases. They basically have Foundations as day zero, AI Fluency as day 30, Scope and Prioritize as day 60, Build and Scale as day 90 And I know that's just a demonstration example to get people thinking in relative terms. But I might actually put Foundations as an ongoing process that happens throughout and around all the other parts of this iterative framework. If you think about just three categories of these Foundations, that leadership team alignment that I was talking about, governance that can evolve as the technology evolves, which is extremely important and very different than some other types of governance structures that we've dealt with in the past, and what I'm calling loosely data improvement, which means a continual improvement of the quality of the data, the readiness of the data, as well as the access and provisioning of the data, which is no mean feat in and of itself. These are things that are not ultimately going to get done once and just be done. They are instead ongoing processes that will continue to shape the relative success or failure of AI initiatives throughout the life cycle of those initiatives. So, of course, we had AIs help to modify the visual slightly to show foundations as a process that happens throughout and around the rest of the work.