Morally Binding AI Governance Isn’t Legally Binding AI Regulation: AI Safety, Muse, and Data Center Financing

Ray Rike (0:08)

Welcome to the AI to ROI Big Story. I'm Ray Rike, Founder and CEO of Benchmarkit.

Peter Buchanan (0:15)

And I am Peter Buchanan. I am the managing partner of NewPlan. And this week, Ray, we're doing something a little bit different than our normal podcast. We are covering four important news stories from the October 2nd edition of AI to ROI News and Analysis Edition. And those four news stories are the Trump administration's voluntary AI safety pact, OpenAI's security crisis, Meta Muse for small business, and the big launch of Muse in general. It's been a total hit record.

And the data center financing warning, the lights just keep flashing red. So, let's go, right?

Ray Rike (0:58)

Okay, Peter. Well, as you know, I've been in Europe for the last couple of weeks on a little bit of holiday.

And the AI industry definitely did not slow down or wait for me to get back. So I'm recording this episode from Brettislava, Slovakia. It's a little bit late here. So if I make any mistakes, I'm blaming time zone and time lags. But before we see if I can catch up, let's hear a word from our sponsor.

This episode of the AI to ROI podcast is brought to you by benchmarkit.ai.

If you're trying to figure out how AI is actually performing inside companies, the real numbers without the hype, Benchmarkit is the resource. They're the industry's leading benchmarking research firm for how companies are deploying AI and measuring the business value it creates. Their research covers AI maturity and best practices across finance, sales, marketing, customer success, operations, and much more. So you can see exactly where your organization stacks up against peers who are further along. And here's the best part, Benchmarkit's research reports and interactive benchmarks are all free at benchmarkit.ai. And while you're there, grab the big book of AI Metrics. It's a free guide built around a five-layer AI business measurement framework that includes 81 AI metrics across 13 functions. If you need a common language for measuring AI to ROI, this is it. That's benchmarkit.ai. Free benchmarks, free frameworks, free content, real data. Now back to the show. Okay, Peter, let's take it away.

Peter Buchanan (2:42)

Okay, so let's first talk about the voluntary AI safety pact that the Trump administration, specifically President Trump, signed with the six major AI companies on September 29th. So there were six companies there who signed it. Google, Anthropic, Meta, OpenAI, SpaceX AI, which is now SpaceX AI or something, and Nvidia.

It's a one-page agreement. It carries no penalties. It sets no timelines or goals or really anything else. The companies who are there have had a bunch of rogue agent incidents recently, and aren't very good at validating their own safety, and the document even misspells the word the United States. So, Semitfor reported that the idea grew out of a conversation between Mark Zuckerberg and House Speaker Mike Johnson during a state dinner. I don't think that's a great way to make policy, Ray. What do you think?

Ray Rike (3:49)

Well, I'd say it's a start, and this is just a first draft. So, I don't think it's, quote, unquote, just a stunt.

The document calls for four layers of control. There's capability and alignment monitoring. There's an external verification team, but also an external auditor that's going to be used, and then board committee oversight. I think one of the things that's really important to note is, I believe it says it may make sense to codify the steps into law someday.

I think it definitely makes sense, but our Congress moves so darn slow. I think this is a good way to get things jump started. And by the way, I think that the board of directors of each of these companies should be demanding some of these governance and oversight controls anyhow. You know, the group of six say that this is just good business common sense and an act of Congress isn't required. In fact, I think President Trump jumped in and said, you know, there's already existing laws that cover AI incidents between the Justice Department and FBI providing oversight. And the big word here, Peter, he called the pattern morally binding. I don't know if morals lead law, right? So I just disagree that this is what we're going to need. But I think maybe it's a good starting point.

Peter Buchanan (5:18)

Right. So and we've run this experiment before. So during the Biden administration, there was a team that created a very similar but longer document that was a Voluntary Pledge. It was written partly by Ben Buchanan, a Johns Hopkins AI professor.

And he was interviewed recently. He said, voluntary pledges aren't enough. And by the way, I'm not related to him. He's much smarter than me because he's at Johns Hopkins. But anyway, so we're going to discuss another news story after this one that shows that self-policing of AI models absolutely hasn't worked. So calling the new PAC morally binding doesn't really mean anything, especially since one of the companies attending the signing ceremony has made a lot of morally questionable decisions related to AI safety and security.