AI safety concerns grow as insiders issue urgent warnings

SPEAKER_1 (0:00)

Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odoo, the only business software you'll ever need. It's an all-in-one, fully integrated platform that makes your work easier, from CRM, accounting, inventory, e-commerce and more. And the best part, Odoo replaces multiple expensive platforms for a fraction of the cost. This is why over thousands of businesses have made the switch. So why not you?

Try Odoo for free at odoo.com. That's odoo.com.

SPEAKER_2 (0:30)

Support comes from Wise, the smart way to manage the currencies you need around the globe. Fed up with losing out to hidden fees when you send money abroad with your everyday bank? Choose the smart way, Wise. You can count on the exchange rate you'd usually find on Google. No unwelcome surprises. Plus ditch that, where's my money feeling? Most transfers arrive in under 20 seconds. Join millions saving billions on hidden fees. Be smart, get Wise. Download the Wise app today. T's and C's apply.

Meghan McCarty Carino (1:01)

The week AI doom went mainstream. From American Public Media, this is Marketplace Tech. I'm Meghan McCarty Carino.

This week, an anthropic researcher publicly resigned, warning on social media that AI companies are gambling with our lives. We'll get into it on today's Marketplace Tech Bytes Week in Review. Plus, data center developers have their eyes on Patagonia, the colder the better. And Amazon is betting AI lip-syncing can make dubbed video content more watchable. But first, back to the AI apocalypse. It's not just one researcher, but a growing chorus of insiders pleading to slow the AI race down. One said he believed there was a greater than 10% chance the tech will kill us all in the next decade.

It comes after a fresh wave of troubling details about the open AI incident where agents escaped a testing environment and hacked into Hugging Face. Here to break it down is Maria Curi, tech policy reporter at Axios.

Maria Curi (2:16)

Some of these researchers, particularly at Anthropic and open AI, have long warned of catastrophic risk and of AI dangers and the whole P-Doom probability. There's different types of guesses and whatnot. But I think what really made this moment different is that this tweet went so viral that I feel like now it has entered the public consciousness in a way that it wasn't before. It used to be a few people that were paying attention to this stuff.

And if the general public was aware of this stuff, it was more so in the camp of data centers. And now I think the actual risks of AI are becoming more evident. And we've spent a lot of time kind of talking about, is this a marketing stunt? Is this something that we should trust these researchers on?

To me, I think the most important thing is that post Hugging Face, you had independent auditors come in and check it out for themselves. People that have no ties or connections to IPOs that are coming up for either one of these labs, people that have no financial incentives at all, they got to go in, test and see and get a look at what happened with the Hugging Face breach. And they said there's real cause for concern here. And so in addition to the public consciousness that this tweet has created, for better or for worse, there's also a lot more motivation on Capitol Hill and Washington to get something done.

Meghan McCarty Carino (3:29)

Yeah, I want to ask you about that. So the public is kind of tuned in here, but what about our leaders in Washington?

Maria Curi (3:35)

That's right. So, I mean, they have been talking about this since 2023

That's when Sam Altman was on the Hill. There have been a lot of hearings. There have been a lot of bills introduced, committees, bipartisan groups. There are a lot of proposals on the table right now. I think now it's a matter of the political will for congressional leadership and for the White House to get them across the finish line.

Usually, when we talk about these debates, there are certain concerns that are immediately flagged. One of them is regulatory capture. If you do some form of regulation, then only the biggest players, Anthropic and OpenAI, are going to be able to adhere to them, then all the smaller players fall by the wayside. There is legislation that has tiered systems so that smaller players aren't required as much as the larger players. There are lawmakers that have thought through these details. And so it's a matter now of whether the political will exist.

Meghan McCarty Carino (4:30)