AI Safety Push Sparks Demand for Watchdog Groups, China’s View of AI Safety, Compute’s Down Payment

Akash Pasricha (0:13)

Welcome, everyone, to The Information's TITV. My name is Akash Pasricha. It is Friday, September 18th. Today on the show, we are covering the big question around whether or not AI safety review organizations are actually independent. We published a piece on that this morning. I'm going to bring on the reporters for a chat. We'll then bring on an expert on China to hear about how we should be thinking about its perspectives on AI safety. We're also tracking new funding in the open weight race. Arcee AI was valued at a billion dollars. I'm going to be talking with the founder in a few minutes. And we're going to close out the show with the editor's cut where we will take a look at the creative ways that CFOs are financing the AI build out. It's going to be a great show, so let's get right on into it.

All the alarm bells around AI safety lately have prompted many to look to watchdog groups for their independent reviews of certain technologies. The problem is that the very independence of these groups is the subject of much debate. My colleagues Rocket Drew and Tiffany Lee wrote about that issue in a new story out today. I want to bring on Rocket for a conversation.

Rocket, welcome back to the show. It's great to have you here.

Rocket Drew (1:23)

Thanks, Akash. It's great to be here. It feels like it's been a while. I'm excited to be back.

Akash Pasricha (1:27)

Well, I mean, you were busy launching your own show. I mean, you got stuff on the go, my friend. You've got stories, you've got a podcast, we've got our AI Agenda Live Summit next week, which you're going to be involved in as well.

Rocket Drew (1:41)

I'm getting excited.

Akash Pasricha (1:44)

You are a busy man. Thank you for making time for us. We appreciate it. Of course.

It was a really great story that you and Tiffany published today. I want to get into it because we've been talking about the AI safety discussion, and I think we understand what the different perspectives are on how to regulate this. Well, that's the central part of this really, is actually not just how, but who should regulate it. You talked about these watchdog groups in your story. Tell us a little bit about who these groups are and why they've gotten a lot more attention recently.

Rocket Drew (2:16)

Yeah, absolutely.

The thing to understand right now is that there's all this interest in pacing the frontier, that the AI companies are for the first time interested in coordinating to slow down the pace of progress in the capabilities of the AIs, so that they can spend more resources on making the AIs safe, sort of buys them time to focus on more safety work.

The problem is, how do you know if you're slowing down? So one of the proposals that's been circulating lately is you should have evaluation groups, basically independent third-party groups that come in and work very closely with the AI companies, and they can help answer that question of whether you're slowing down, whether you're being appropriately safe. So Dario Amadei unilaterally committed Anthropic to taking some of these steps, and then Sam Altman from OpenAI jumped in and said, OpenAI is interested in doing the same. The advantage of bringing in these third parties and embedding them within the AI company is that they can add more transparency so they can tell the public about what's going on. They can verify the statement that the AI company is making. So when the company says, our technology looks safe to us, the evaluator, the independent third party can say, it looks safe to us too, and they can offer a second opinion internally on the decisions that the AI company is making. So those are the advantages that Dario Amadei pointed to. So then the question comes up, who are these organizations going to be? Everyone's looking around the industry, and there are some natural candidates, because there are some independent third party groups, or supposedly independent groups, that have worked with these companies for a long time. At the same time, other organizations are throwing their hands up and saying, well, I think I should be the independent third party watchdog. So there's more and more interest in this.

Akash Pasricha (3:59)

Everyone's like, I'm the one, I'm the chosen one.

Rocket Drew (4:02)

Right, pick me, pick me. Yeah, exactly.

Akash Pasricha (4:06)

So who are the groups that are, I mean, who are the big names that we should be paying attention to?

Rocket Drew (4:10)

Yeah, so right now, when an AI company, Anthropic, OpenAI, even Google, Meta, when they're going to release a new AI to the public, they want to know what it's capable of. And they especially want to know if it has any dangerous capabilities. And that means exactly what you would expect. It means, is this thing going to be good at cyber attacks? Is it going to be capable of creating a weapon of mass destruction or at least helping someone create such a weapon? And the AI companies themselves don't always have the expertise to answer that question. So that's when they bring in these outside evaluators.