AI liability for MSPs: missing policies create audit risk — Ignacio Lopez, Work-Smart.ai

Dave Sobel (0:01)

When an AI system hands back, a bad answer, what happens next? How's that audited? What questions need to be asked? Ignacio Lopez is a former SOX Auditor, now AI Developer and a former Kaseya Product Manager who understands this space. He wants to help out his clients with AI, and we talk about what that means for them and for MSPs.

This is the Business of Tech.

Ignacio Lopez, you are the Fractional Head of AI at worksmart.ai.

Welcome to the Business of Tech.

Ignacio Lopez (0:38)

Thank you for having me.

Dave Sobel (0:40)

I'm excited to talk to you. You build AI inside places like family offices, law firms, construction groups, and we know nearly all of them run working with an MSP. So, when you walk in, what is the MSP already put in place for AI?

Ignacio Lopez (0:59)

So I'm going to be honest, not much. So my work starts with an assessment to understand if they are ready to implement AI, and then I work in conjunction with the MSP to deploy, configure, and run the AI tool they decide to roll out.

Dave Sobel (1:19)

Okay. Now, is the MSP usually in the room for the engagement? Do they find out later? Like, how does that work?

Ignacio Lopez (1:25)

So, to be honest, they are not engaged from the beginning, and I feel that's something that needs to change. The MSP should be involved from the beginning, because we are talking about companies that are regulated, that they need to present evidence like any other company. So MSP should be involved from the beginning. But in my experience, I'm working with mainly the wealth management firm, construction firms.

They are not involved at the beginning. I have to involve them. I have to ask them to do some things, but I think they should be involved in the definition plan, like the step zero.

Dave Sobel (2:03)

So, how many of the firms had anything in writing about approving AI settings or configuration? Like how many policies? Like what's the landscape look like when you walk in?

Ignacio Lopez (2:13)

Pretty terrible, I will say. So, I will say like only 20% of my current engagement had an AI policy written, and not only written, because one thing is to have a document written, but then you need to communicate it to your company, to anyone who is using AI.

And not so long ago, I did a webinar for family offices. There were 53 different family offices, and 80% stated, I did a poll, that they were using AI without a written AI policy. So, that tells you how the market is doing in terms of AI and governance.

Dave Sobel (2:51)

It's pretty bad out there right now. It is, it is. So, one of the things I also know, you've got a background in some of the insurance side. You started your career on the auditor side of the table. And one of the things that's really interests me, like since January, insurers can attach generative AI exclusions, things like Cg4047 at renewal.

Now, you've got experience around this. When a claim or an audit lands, like what evidence does the auditor or insurer like accept? And how does it compare with the logs most MSPs are keeping right now?

Ignacio Lopez (3:22)

So, just to give you a bit about my background, I was an editor, a SOX editor for KPMG, and I work as an information risk manager in JP Morgan. So, we are talking about one of the biggest banks in the world. So, it was highly regulated. So, I learned what an editor wants to see and how to produce the right evidence. So, nowadays, it's not just about having a log.

That's just a small piece. What an editor wants to see is the evidence that there's a process in place that was discussed and approved, and that you have the right controls in place, and that you can produce evidence that shows that you have the process, that you have the control, and it was done, and it's dated, because what I see on a lot of my clients, they have some logs, but they don't have it dated. And an editor wants to see what happened in March, what happened in April, so you can produce evidence or logs from March in September. They need to be able to trace back from the logs the controls you have in place to enforce what you approve, like your policy. They want to see the full cycle of the process.

Dave Sobel (4:36)

Well, that makes sense. Now, the other thing I want to talk a little bit about is, so in SOCKS terms, you test whether a control was designed right and whether it actually operated.