The Human in the Loop: The Skill You Control

Carlo Thompson (0:05)

Welcome to Surviving AI with Carlo Thompson. I'm Carlo Thompson, and this week, we're going to talk about Human in the Loop.

Four in 10 workers are already using AI at work, and no more than a quarter of those workers got any training on how to use these tools.

Ainsley (0:30)

Artificial.

That is the whole episode, honestly. Four in 10, give or take, Pew's got it at 38 percent, Gallup's up near 52, depends who's counting and how. But the training number barely moves no matter who asks. New York Fed found under 16 percent of workers say their employer offers any AI training at all. So you've got nearly half the workforce using this stuff, and the company mostly just handed it over and walked away.

Carlo Thompson (1:14)

The thing that I think I want to start with, Ainsley, is this idea around training in general.

Because the prescription of training relative to this new thing is somewhat hard to get to. So I'm not going to hold corporations to this idea that they may not know what to tell people to go get trained on.

But they could potentially give people an idea of, hey, you need to go get training on X, Y, Z. Here's a thought process that I'm thinking in my head, is that you have some function in the organization, the main experience that you've been doing for however long you've been doing it, white collar worker. Then now we have AI.

They may not be any specific tool that's directed toward your domain yet. However, corporations are just blanket out there saying, hey, use AI. And mostly that turns up as here is this AI chat bot incident that you could go work with and create skills and create things to help you gain efficiency in your job, right? Write your email, this, that and the third thing. There's nothing really out there suggesting that, you know, maybe there is. Sorry, I had started that out wrong, but there's things out there specifically around tools like Harvey AI, for example. Harvey AI is a legal tool. Now, if you're in the legal profession, it's pretty easy for a corporation to come, come by and say, hey, Harvey AI is with us now, go get training on Harvey AI. But it's harder for some other titles out there in the world. And we kind of got to rationalize that, rationalize that before we get into the whole episode about Human in the Loop.

Ainsley (3:03)

That's actually a really useful split because use AI and use Harvey are not the same instruction at all. One tells you which tool and implicitly what it's supposed to catch or miss in legal documents. The other is just a login. And I think that's why the training numbers look so bad across the board. Gallup's other finding here is only about a quarter of workers say the company has even communicated a clear plan for AI. Forget training.

So for the domain specific tools, the failure modes practically come preloaded. For everyone else stuck with a general chatbot, they're the ones who have to reverse engineer where it breaks with zero help.

Is that the divide you're drawing? Tool specific guidance versus generic access?

Carlo Thompson (3:55)

I think it is, right? Because it really comes down to, you know, this idea around human in the loop and specifically to what, you know, folks are going to busy themself on doing, right? Because when you mention this idea around human in the loop, right? Really what you're suggesting is that you're in the loop of whatever AI output is coming out and then you now as a human has to go review that to make sure that you find the right failure modes, right?

So with Harvey AI, with legal folks, your leadership could point you straight at that tool and say, go get training in Harvey AI to figure out where the failure modes exist. But for everybody else that don't really have a tool per se, they have to learn by using the tool and gaining the repetition that they need to gain to get to the strength in how they could discern the output, whether it's right or wrong, based on their domain experience.

Ainsley (4:58)

Right. So the domain expertise has to do double duty for everyone who isn't a lawyer with Harvey.

And here's the part I'd push on. There's actual research on this, going back to aviation and clinical settings, that says experts overtrust automated output just as much as novices do. Training alone doesn't fix it. What actually helped in those studies was pairing practice with real accountability, knowing you're the one whose name is on it if it's wrong. So the repetition you're describing only works if something's forcing you to actually check, not just skim and approve.