AI for MSPs: Nobody Is Counting the Cleanup

Dave Sobel (0:02)

OpenAI just handed mathematicians 700 papers and a warning that some of them could have issues. The checking is now somebody else's job. For your client's AI, that somebody is you. This is the Business of Tech. I'm Dave Sobel. The AI industry has a new habit. Ship the output, skip the part that makes it usable. Start with mathematics. OpenAI posted 722 mathematics manuscripts to GitHub, grouped into 372 results and produced by an internal model the public can't use. OpenAI's own repository warns that some of the results that haven't been formally checked, quote, could have issues, end quote. By the verge's count, only about 4 in 10 of the main results come with a machine checkable proof. Mathematicians told the verge it could take years just to understand what's there. A referee rating one paper a day every day would need nearly two years to get through them. The field had asked for something different. Weeks earlier, 25 Fields Medal winners signed an open letter warning that AI Labs, racing to solve famous problems, were threatening the work itself. Their concern was solutions announced without proper write-ups, without citation of prior or without anyone able to explain them. OpenAI withdrew its sponsorship of a Caltech math event after researchers there criticized it. And a week before the release, an independent advisory group of mathematicians at the Institute for Advanced Study, one OpenAI itself consulted, asked the labs to disclose the model, the prompts and the compute behind each result. OpenAI didn't name the model, published no prompts, and gave only compute as an average. The people downstream felt it immediately. NYU mathematician Tristan Buckmaster told Business Insider that entire research programs were wiped out in a day. Early career researchers had built grant proposals around problems that now have claimed answers. OpenAI says the release includes a process for revisions and citations and plans to fund workshops to help researchers work through it. Some of it may well be real. That isn't the question. It isn't only math. In its own report on Unintended Model Actions, Anthropic disclosed that during testing, Claude Haiku 4.5 filled in a Philadelphia police tip form about a murder case with no name and no contact details. I may have information regarding this case, it wrote. Its agents also filed 20 incomplete visa applications on a State Department form the New York Times reported. None were processed. Anthropic says the cases had minimal real-world impact and it disclosed them itself. But a police department still had to deal with a tip that came from a test. Different fields, same shape. The output shipped. The preparation the receiving side depends on didn't. So why does the preparation keep getting skipped?

If you're listening to this and haven't hit follow yet, on Apple Podcast Search Business of Tech. It takes 5 seconds and you'll get the next episode automatically.

Every MSP I talk to has the same problem. They've got AI. What they don't have is a way to charge for it. Pax8 calls the fix Managed Intelligence, selling outcomes customers want to pay for, not just one-off tools. The Pax8 Marketplace is where you build it. Stop reselling software. Start running intelligence. 47,000 MSPs already made the move. Start yours at pax8.com. That's P-A-X, the number eight ,.com.

Every one of those norms exists for the same reason. They make whoever produces the work carry the cost of checking it. Peer review asks an author to write a result up so others can follow it. Responsible disclosure asks a bug hunter to confirm the bug before filing it. That arrangement held because producing the work was expensive too. Nobody wrote 700 papers in a month.

AI broke that balance on one side only. Generating got nearly free. Checking didn't get any cheaper. You can watch it happen in open source. Linus Tovold told kernel developers that the flood of AI reports has made the Linux security mailing list almost entirely unmanageable as the register reports. Different people run the same tools, find the same bugs and file the same reports. Maintainers spend their time forwarding them or pointing out that the fix shipped weeks ago. Tovold calls it pointless make-believe work. He's asked to reporters is the old norm, restated. Read the documentation, write a patch, add something the AI didn't.

Now look at what happens when a producer actually counts. Anthropics free OSS scanner sends model written vulnerability reports to the open source projects that sign up with no human review. And Anthropic warns some will be wrong. Anthropic also publishes a dashboard for its broader disclosure program. By its own numbers, it has disclosed 6,157 vulnerabilities across 591 projects. Maintainers acknowledged 5,103.

516 have been patched upstream, about 1 in 10 Even so, the finding was the easy part. The fixing is still on the maintainer side of the ledger. And Anthropic itself notes that a released fix doesn't mean it's installed. Every day between found and fixed is the day the bug is known and still open.