The Real AI Risk Facing Banks and Credit Unions

Jim Marous (0:00)

I was in a board meeting this week when the director asked whether their institution should slow down its AI efforts because of everything he'd been reading about in the news. He won't be the last one to ask.

63% of Americans now say AI is advancing too quickly. 2% say too slowly. And the people building the most advanced systems in the world have begun warning about the pace themselves.

That concern deserves a serious answer, but it doesn't give us permission to stop or even slow down. Months before any of this noise, our industry had already identified their greater risk.

The American Bankers Association asked large banks and community banking leaders how they were handling AI.

Plenty described themselves as cautious, but most said doing nothing was the bigger danger, and they were very specific about why.

Growing dependence on vendors, expertise that never develops inside your own building, and ground lost as AI becomes part of everyday banking. Nothing in the last several weeks has made any of those any less true.

Here's what that director had been reading. The heads of the largest AI labs have all publicly called for slowing the pace of AI development. And OpenAI published six cases of what it called unexpected or concerning behavior from its own models, along with a framework for reporting future cases. Almost no one in our industry is reading those six disclosures. I think we should. One of those models wrote jailbreak style instructions into its own notes, so it could bypass the limits it had been given. Others used an internal system as a message board, passing notes to each other while working the same problem. Read the reports, and what's described is pretty familiar.

Systems rewarding the wrong things. Tasks that couldn't be completed so the thing went hunting for a workaround. Errors nobody wrote down at the time. We've all seen that movie. It's the sales incentive that hits its number but damages the relationship. It's the referral contest that filled the pipeline with garbage. The technology is new.

That particular failure is the one we built plenty of times ourselves, which tells us what to ask about anything already running in our own shop. What did we tell it to optimize and what behavior does that reward? Who's watching it?

Where do the errors get written down and has anyone read them? Back to the director's question. The debate and decision within your institution are connected, but they're not the same direction. Those labs are deciding how quickly to build more powerful models, work that requires advanced chips, enormous computing power and electricity run them.

We're deciding where to use the existing models, what level of access to grant them, and who remains accountable. That's a governance question, and those reports have more to say on it than the slow down debate does. Our customers have already set their own boundaries. TD Bank surveyed more than 2,500 consumers this year and found that 55% now use AI to help manage their money up from 10% a year ago. Roughly two-thirds are comfortable with it, working behind the scenes on fraud detection, spending tracking, and recommendations. Only 18% would let it make an important financial decision on its own, and 81% want a person involved when they call us for help. They're comfortable with AI helping them understand and protect their money. They're far less comfortable about letting it decide anything that matters. So that's a usable map and it's more specific than anything in the headlines. Most of us will never formally decide to stop. When the environment gets this loud, the threshold for waiting drops. The AI recommendation just moves to next year's plan. The update we send upstairs says we're monitoring the situation. Nobody blocks anything. Nobody argues, but a year goes by. And in that year, three costs, our industry already named, show up. The vendor starts making decisions for you that you deferred, and you take whatever they built. Your people don't develop the judgment to evaluate any of it because nobody builds that by reading about it. And the institution that kept moving gets further ahead. Most of us are writing next year's plan right now. So here's where I'd place it. Deploy now, where AI supports one of your people or protects the customer without acting on its own. Fraud detection and scam alerts, where the technology is proven and the loss numbers carry the business case.

Document handling in lending and account opening, the extraction and exception work that still moves at human speed. Contact center assist, pointing at your employee rather than your customer, where rep edits what the machine drafts. TD respondents show broad comfort with AI working behind the scenes, particularly for fraud detection, activity tracking, and making recommendation. The rest of this list is my own judgment and where I'd start. Pilot now where the customer can see it. Personalization and next-bex action is the one I'd pick first because our own research found that just 6% of institutions have adopted it while it sits near the top of everybody's priority list. That's one of the biggest gaps in our research between what we say matters and what we're actually doing. Run it in one product line, measure it against a control group, and keep a person in the approval path. And not yet for AI that decides on its own. I don't mean the automated underwriting and the rules-based fraud decisioning we've run for years, which are governed, documented, and working. I mean the system that can move money, change a product, or makes a consequential credit call outside the rules you wrote with no approval step in the middle. Again, 18% of TD's respondents would let AI make an important financial decision on its own, and 81% want a person involved when they call us. That's the market telling us where the line sits today. And that line will move as trust grows, but right now, customers are drawing that line. If you want to see where your institution sits against everyone else on this, our 2026 Retail Banking Trends and Priority Reports is a free download at digitalbankingreport.com and the links in the description below. It found that 62% of institutions have been working on digital transformation for more than two years, and only 7% have fully achieved what they set out to do. We've known what to do for a while now, so should your institution actually slow down on AI?