Google DeepMind's AI Autonomy, Anthropic IPO Voting Control & the New AI Safety Group

Akash Pasricha (0:13)

Welcome, everyone, to The Information's TITV. My name is Akash Pasricha. It is Friday, September 25th. Today on the show, we have a busy show for you. We have some highlights for you from our AI Agenda Live conference. We've got our conversation with Google DeepMind's Chief AI Architect. We're going to play a clip for you shortly. We also have some new reporting on a regulatory body that Google, OpenAI and Anthropic are all working on. We have the latest on DeepSeek's funding round out of our Asia Bureau, and we're going to bring on the author of our latest op-ed, who is arguing that America's AI ambitions are falling short of what they need to be to remain competitive.

We're going to wrap the show with a look at The Information's new AI Inflection Index for Pro subscribers. It's a busy Friday, so let's get right on into it.

One of the standout conversations at our AI Agenda Live conference this week was our Editor-in-Chief Jessica Lessin's interview with Google DeepMind Chief Architect Koray Kavukcuoglu. I want to play a clip for you of Koray giving us one of his closest looks yet on his view on recursive self-improvement. Here is that conversation.

Jessica Lessin (1:27)

Are the models training themselves yet? Have we reached this recursive place and if not, when?

Koray Kavukcuoglu (1:36)

When we talk about RSI, when we talk about recursive self-improvement, I think the first thing that comes to everyone's mind is, oh, there is a system that just end-to-end trains itself, and without any human intervention, there's something that is happening in that.

Jessica Lessin (1:51)

Kind of in the name, but yes.

Koray Kavukcuoglu (1:53)

Right. But I mean, you can think that that set up is not necessarily right now end-to-end the whole system. We are not talking about that. There are stages before you get to that. The first thing that already has happened is, of course, human in the loop, we are using our own models, we are coding with our own models. That gives us speed, that gives us the capability to research more, to explore more. That has happened. With that, we are partnering with AIs to do research better. The second stage from there is, when you think about it, as we talked, the training process, it's like a six-month process.

When you think about, okay, there's a model development going on, it's not like a two-week thing. It is like, we might release models every two to three weeks, but that means that you have a big overlapping windows of training that is going on. In that long training horizon, there are pockets of times where we will trust agents to actually do the training.

And autonomously go and make experiments, look at the results from those results, figure out new experiments and new hypotheses, and run those experiments, and then come up with a solution that is all guided and supervised by humans. That is where we are right now. Of course, that is another level of partnership with the agents. And I think we are in the steps through that, but I wouldn't say when we say RSI, but I think people imagine this whole big system.

Juro Osawa (3:21)

You press a button and you have...

Koray Kavukcuoglu (3:22)

Right, yeah, exactly. I don't like...

Jessica Lessin (3:24)

So, six months ago, were you... Were agents involved in the same way? I'm just trying to get a sense of what curve we might be on.

Koray Kavukcuoglu (3:32)

No, but that's the progression that is happening. Six months ago, I think all of us were talking about, oh, we are using our own models to do coding, we are using our own models to do research, that you have an idea, you would communicate with an agent, you would like explore ideas, like bounce ideas so forth and those kinds of things. Now, like you are trusting the agents more. That's the important bit. What we are trying to do is we are trying to build intelligence collaborators, co-workers, that you can trust. The more you can trust that, the more you are going to be able to let that model, that agent run experiments, but it's again in coordination, in communication and like under that supervision.

Akash Pasricha (4:18)

For more coverage of the conference, I want to bring on Stephanie Palazzolo, who hosted the event and many of the conversations on stage. Stephanie, welcome back to the show. It's great to have you here.

Stephanie Palazzolo (4:28)

Great to be back, Akash.

Akash Pasricha (4:30)

Let's talk about the highlights from this week's event for you. What were some of the themes that really stood out to you from your conversations on stage?