So Could AI 'Kill Us All'?

SPEAKER_1 (0:02)

Bloomberg Audio Studios, Podcasts, Radio, News.

Sarah Holder (0:10)

Earlier this week, an AI researcher at Anthropic resigned from his job and posted about it on X. He'd spent the last three years working at both Anthropic and OpenAI, he wrote, and neither company is acting responsibly, he said. He said the companies were gambling with our lives, and suggested that people in the AI space thought the technology could eventually kill us all.

It was scary stuff, and the post made waves in the AI industry and beyond at a time when concerns about the risks of AI are mounting.

SPEAKER_3 (0:41)

Bill Gates says the computer industry that he championed is now a global threat.

SPEAKER_4 (0:45)

OpenAI system was able to hack into another AI platform called Hugging Face during a security test.

Sarah Holder (0:52)

With me today is Bloomberg's Mike Shepard, Senior Editor for Technology and Strategic Industries. Mike, how would you describe this moment for AI? Are we at a turning point?

Mike Shepard (1:02)

We talk about things that go viral. This took off, but it also has some staying power.

It didn't just ripple through Silicon Valley, it echoed across Washington and in global capitals too. There is this moment of reflection now in this technology that has so much promise to change the economy and change the way we live.

Sarah Holder (1:25)

I'm Sarah Holder, and this is the Big Take from Bloomberg News. Today on the show, the AI safety reckoning is here.

Mike, I am curious why this particular post from this particular researcher broke through though. AI researchers have been sounding the alarm for a while on these kinds of risks. This person, Jacob Coxon, not a public figure, not someone I'd heard of before. He doesn't have any other tweets on his profile. Why did people pay so much attention to what Jacob Coxon was saying and what did he actually say about why he left the company and the things that he's so concerned by?

Mike Shepard (2:02)

I've been reflecting on this too.

One, its simplicity. It was very direct. It was personal. He really addressed the audience, not just a tech audience, but really a Main Street audience as well.

He wasn't very preachy in it either. He was issuing a warning, but he left it to the reader, the consumer, the audience, to try to make their own decisions. He did issue a warning that in his view, after having worked at both Anthropic, which he just resigned from, and OpenAI, his prior employer, that neither company was acting responsibly in their respective pursuits of what's known as super intelligence. This is the version of artificial intelligence that surpasses human capabilities in most areas, and also possesses the ability to improve itself. He called that pursuit of what's called recursive learning or self-improvement, he cast it as reckless, really, Sarah. He spelled out in passing the cybersecurity risks, that there could be a massive and crippling hacking attack using AI that could really disrupt financial systems and critical infrastructure. There's also the threat of bioweapons too. So, all of this kind of hangs in the background of his message. But then he also issued an admonishment to his fellow Silicon Valley AI workers. Like, hey, do you really want to be a part of this? Are you sure you want to have a role in developing this technology in a way that could be so destructive and consequential in a negative way?

Sarah Holder (3:44)

I put that question to you, Mike. Why are people still building it?

Mike Shepard (3:48)

Well, for Anthropic, part of their identity is to pursue AI development responsibly. And they actually see themselves as the company that can achieve this super intelligence or artificial general intelligence in a way that will be safe for humanity. And it's not just that Anthropic, a number of researchers out there are committed to the pursuit of this technology for that reason. They want to get there first so that it's done safely. So they believe in it. Others are a little bit more skeptical about the apocalyptic vision that some are casting as the backdrop for AI development, that it will kill us all. And they think, hey, look, it's not that bad. We can control it.

We can understand it. And most AI will actually be used in much smaller and refined applications.

Sarah Holder (4:42)

I want to drill into this idea that AI will kill us all, because, you know, Jacob's former colleague, an alignment science lead at Anthropic, reposted Jacob Cox's post and said that, quote, we really do earnestly believe AI could kill all humans, unquote. And he gave it a more than 10 percent chance within the next decade. That's like a pretty remarkable thing for someone who currently works at Anthropic to say, right?