Insightful and Workflow Analytics: Measuring Activity vs. Actual Value for MSPs

Dave Sobel (0:02)

A manager opens a report showing how much the team uses AI, the kind of report an MSP can now resell at a 30% partner discount. Nearly half of workers admit they exaggerate exactly that number. And in McKinsey's research, barely one leader in 10 says their organization has rebuilt its work around AI, the kind of change that would give the report something real to measure. This is the Business of Tech. I'm Dave Sobel.

The channel has decided AI is its next big revenue line. The evidence about what makes AI pay hasn't caught up. We'll start with Informa's MSP 501 survey. More than 600 managed service providers. Fifty-seven percent expect AI to be among their biggest revenue gainers over the next 12 months. A year ago, that number was 37 Ninety-one percent now offer or use AI. And about a third expect their AI revenue to grow by more than 20 percent. Only managed security ranks higher. Now look at how those same providers describe their own use of it. Fifty-four percent name employee productivity as a key internal use. Thirty-five percent name reducing repetitive work. Those are tools in people's hands. They are not a business that runs differently. That difference is the one IHL group president Greg Busek put in front of channel partners at Blue Star's Vartech event as Managed Services Journal reports. He cited McKinsey Research. Only 11 percent of leaders surveyed said their organizations had reached what McKinsey calls reinvention, redesigning roles, workflows and operating models around AI.

In that group, 48 percent report realizing enterprise value. Among companies that simply give employees AI tools, it's 13 percent. The ones that rebuilt the work report value nearly four times as often. And McKinsey notes it deliberately recruited more advanced companies, so it warns that 11 percent isn't a market-wide estimate. Even in a sample tilted toward leaders, about 1 in 10 did the redesign. Nearly half of those got value. Most of everyone else didn't. The chief executives say the same thing from the top. PwC's annual survey of 4,454 CEOs across 95 countries found 56 percent have seen no significant financial benefit from AI so far. 1 in 8 said it delivered both lower costs and higher revenue. Those were 2 to 3 times more likely to have embedded AI extensively across the business. So put the 3 side by side. The channel's AI forecast is climbing fast. The redesign that produces measurable value is rare, even among leaders McKinsey went looking for. And most CEOs can't point to a return. The forecast is built on adoption. The returns show up where the work was rebuilt. Those are not the same number.

So if the returns come from rebuilt work, what are all those AI programs actually measuring?

If you're listening to this and haven't hit follow yet on Apple Podcast Search Business of Tech, takes 5 seconds and you'll get the next episode automatically.

This episode is supported by MailProtector. One of the things I keep hearing from MSPs is that they want email security from a vendor focused on MSPs, not a relationship where you're wondering what the roadmap looks like after the next acquisition, or where you fall behind enterprise clients. MailProtector is independent, partner aligned, and to reduce the ticket friction that eats up their day. In a market where trust matters more than ever, that's a real differentiator. Visit mailprotector.com to learn more.

When the work hasn't been rebuilt, there's no new outcome to point at. So leaders point at the one thing they can see, whether people are using the tool. That substitution is the whole problem. A redesigned process gives you a result to measure. A proposal out in a day instead of a week. A ticket closed without a second touch. A tool handed to the old process gives you a usage number. And once usage is what leadership watches, usage is what people produce. McKinsey reaches for an old comparison. When factories first swapped steam engines for electric motors, they kept the same floor plans, the same line, the same management, and productivity barely moved. The gains came only after they rebuilt the factory around the motor.

Visors surveyed a thousand full-time US workers on exactly this. Visors sells workforce analytics software, so weigh it with that in mind. 48% said they have exaggerated their AI usage or expertise to colleagues or leadership. 45% said they feel pressured to use AI even when they're not confident using it well. Nearly 60% are unsure what their employer's plan for AI is. Visors' own conclusion is that the driver of what it calls performative AI use is a lack of trust in leadership.

Pressure to use it, no plan for what the use is for, and nearly half of the people surveyed shading the number. That's what a usage target produces. You might reasonably say that some usage is inflated, but most of it is real, and real usage is still productivity. That's the assumption Forrester takes apart. Writing about content and creative teams, a Forrester analyst describes AI making an individual employee more productive while making organization less efficient. The illustration If AI saves one employee 20 minutes but creates 30 minutes of work for others downstream, that's not a productivity win. The first phase of adoption, the analyst writes, was about individual productivity. The harder phase is the whole workflow, including the work AI created for everyone downstream. So even honest usage measures the wrong thing. It counts the 20 minutes at one desk and misses the 30 at the next. In plain terms, without a redesigned process, there's nothing to measure but activity. Activity gets performed when it's watched and it misleads even when it isn't, which makes the tool that watches activity the easiest AI product there is to sell. And somebody now sells that tool to you, so you can sell it to your clients.