Trump and tech leaders agree to voluntary AI safety accord

SPEAKER_1 (0:00)

This Marketplace podcast is presented by Tomorrow's Cure. If our episodes on the intersection of AI and healthcare left you wanting to learn more about how technology is transforming medicine, Tomorrow's Cure is for you. It's the chart-topping 2025 Ambioward Finalist Podcast from Mayo Clinic. Back for a brand new season with new host award-winning journalist Lindsay Sievert, Tomorrow's Cure explores the innovations changing the healthcare landscape. Featuring conversations with leading physicians, researchers and medical experts, the new season examines everything from AI-powered diagnostics and cutting-edge cancer therapies to surgical technologies improving patient care today.

Not sure where to start? Listen to the season premiere featuring MD Anderson radiation physicist Dr. Paige Taylor and Mayo Clinic radiation oncologist Dr. Adam Holtzman. They discuss why carbon ion therapy is generating excitement in the medical community and what it could mean for the future of precision cancer treatment. So go ahead, follow tomorrow's cure on Apple Podcasts, Spotify, or wherever you get your podcasts.

Megan McCarty Carino (1:05)

Is the Trump administration playing good cop, bad cop with the AI industry? From American Public Media, this is Marketplace Tech. I'm Megan McCarty Carino.

While the White House hosted tech luminaries this week to sign a voluntary agreement on AI safety standards, the Federal Trade Commission opened a broad investigation into AI labs over potential safety threats their products may pose to consumers. We'll get into it on today's Marketplace Tech Bytes Week in Review. Plus, a nearly four-hour documentary on Elon Musk is stirring up controversy over whether social media platforms will run ads for it. But first, back to what President Trump called a morally binding, if not legally binding accord on AI safety. It comes after calls to slow down AI development over concerns about losing control, mainly from Anthropic CEO Dario Amadei.

Here to discuss is Will Oremis, staff writer at The Atlantic.

Will Oremus (2:14)

I think that Anthropic and to maybe to a lesser extent, Open AI are interested in something more binding. There has been a lot of pushback to that from NVIDIA, the chip-making giant, and from Meta, which also has a hand in the AI industry and has placed more of its bets on open-source AI which may be more affected by regulations. They managed to hash out something that everybody agreed to, which I'm a little surprised by. I should note that one of the other people who is against regulation of the industry is the president himself.

There's been reporting from Politico that suggested that Mark Zuckerberg was actually the first to float this idea for the accord to House Speaker Mike Johnson. From there, they got NVIDIA's Jensen Wong involved, and they managed to get Anthropic and Open AI on board as well. That maybe becomes a little less impressive when you realize that it doesn't actually have any teeth other than the morally binding component which the president mentioned.

Megan McCarty Carino (3:15)

Yeah, I know Zuckerberg had talked about, like, this is just a start. Maybe this could be, you know, sort of made into law at some point. But yeah, you could sort of plot, I think, the influence on this agreement from the seating chart around President Trump. So it was sort of, you know, Elon Musk and Jensen Wong flanking the president, Mark Zuckerberg one layer out, Dario Amadei way down the table. But yeah, this is, you know, this is a compromise. This is, stops short of certainly what Dario Amadei had proposed in his Pacing the Frontier essay. Unclear to me if, you know, the Dario Amadei and, you know, OpenAI are still going to move forward with kind of what they informally agreed to on Twitter, I guess.

Or if they're going to also kind of go with what this agreement requires, which is, you know, kind of setting up internal systems for safety, you know, some kind of an independent audit of those systems, board level oversight of these things, but not necessarily the kind of intensive, you know, bring in independent evaluators, slow things down, stop short of that, I guess.

Will Oremus (4:30)

Yeah, it does. Yeah, we used to have handshake agreements in business, and now I guess we have Twitter agreements, or X agreements, as the case may be. I think that, you know, I think this might be low-key a sort of win for Anthropic. I mean, they're the ones who sent out this memo saying we really need to slow things down, we really need to think about safety, you know, this stuff could kill us all. And whether you believe that or not, they have really staked their position on the idea that there needs to be cooperation, that it needs to be not just them. And they seem to be getting some of that. So I don't think it was ever realistic to expect that we would immediately get heavy handed regulations of the AI industry. I don't think Anthropic ever expected that. There was a moment pretty soon after Amode came out with his memo that it seemed like, you know, he was getting piled on, like everybody was jumping on Anthropic, calling them do-mers, dismissing the idea of AI safety.