AI Insiders Keep Saying We’re In Danger — Where’s The Evidence?

SPEAKER_1 (0:01)

Support for the show comes from VCX, the public ticker for private tech. The US stock market started history's greatest way of a wealth creation, from factory workers in Detroit to farmers in Omaha.

Anyone can own a piece of the great American companies. But today, our most innovative companies are staying private longer, which means everyday Americans are missing out. Until now. Introducing VCX, the public ticker for private tech, now available wherever you buy stocks. Visit getvcx.com for more info. That's getvcx.com. Carefully consider the investment material before investing, including objectives, risks, charges and expenses. This and other information can be found in the Fund's Prospectus at getvcx.com. This is a paid sponsorship.

SPEAKER_2 (0:46)

From the Goldman Sachs trading floor in 10 minutes or less, investors and analysts share timely analysis on the week's market activity.

The Markets Podcast from Goldman Sachs. Listen now.

SPEAKER_3 (1:00)

You know that feeling when too many things fall through the cracks? monday.com was built for that gap. The AI work platform where people and agents work side-by-side to deliver more together. Create your first Monday agent today at monday.com.

Ed Elson (1:31)

Welcome to Prof G Markets. I'm Ed Elson. It is September 16th. Let's check in on yesterday's market vitals.

The major indices fell ahead of the Federal Reserve's interest rate decision, which comes out later today. Brent crude topped $109 per barrel. The yield on 10-year treasuries climbed to its highest level in almost two decades. And finally, Bitcoin fell after the Senate blocked the Clarity Act, which proposed definitions and regulations for digital assets. Democrats said the bill did not do enough to address the ethics of President Trump's crypto business.

Okay, what else is happening? Two months ago, a Google DeepMind safety researcher quit, and this week he told everyone why. In a post on X on Monday, Bilal Chugtai announced his resignation from DeepMind where he worked on safety and alignment research. He also wrote, quote, I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome. Chugtai is now at Blue Dot Impact, a non-profit that trains people in AI safety. He is now the second safety researcher to go public about the risks this month, following Jacob Cox's viral resignation from Anthropic last week. Meanwhile, the AI labs are responding with the development of a new regulatory body themselves. Anthropic, OpenAI and Google DeepMind have discussed building an industry coalition to test frontier models before release. OpenAI is also reportedly urging Washington to move forward with a federal AI framework, however, Trump does not appear to be open to that idea in the slightest. So here to discuss all of it, we are speaking with Ed Zitron, host of Better Offline and author of the Where's Your Ed At? Newsletter. Ed, good to see you.

I want to just get your reaction to this whole debate in its entirety, going from the beginning where we had this tweet from Jacob Cox in which he's saying, mega, mega viral, saying that he believes that AI might kill us all by the end of the decade. Then, of course, another anthropic researcher cosigned that statement, said that there was a 10 percent likelihood of it happening. Now, here we are. It's become a national debate, a sensation around the world. What is your take? What do you make of all of this?

Ed Zitron (3:55)

We've now heard at least 11 of these guys say they want a slowdown. We've heard exactly zero of them say what that means. While Dario Amadeus spelled out what he would do, it mostly came down to having METR, which he claims is an independent research organization, despite it being basically funded by Anthropic itself, being an internal auditor, and then some vague stuff about China. These companies not slowing down, none of this safety stuff matters to them. All of it is lip service to a media industry that doesn't actually think for itself. Let's start with Jacob Coxon, though, and why he is the way he is. It could be a cynical grift, it could be that because you'll notice that he doesn't actually explain what it is he's scared of, what is the Anthropic or OpenAI did that was so scary. And indeed, when describing the hugging face attack, for example, Anthropomorphizes it and never holds the AI labs accountable. It's always, oh, AI is this unknowable thing. But it starts with the rationalists and effective altruists. These are, while they will describe themselves as people that are rationally thinking about the dangers of superintelligence, what they actually amount to is a kind of religious cult.

These people are a large part of the AI labs, and they spread this idea that AI will inevitably, foregone conclusion, definitely kill us all unless we stop it. But stopping it always involves giving a specific corporation they like money and power. So there is a section of these people who truly believe this. They believe it on very flimsy terms. They don't really have any evidence for it other than going, look, a thing we vaguely predicted kind of happened not exactly the way.