What Happens When AI Starts Improving AI? | TITV’s AI Deep Dive

Noam Brown (0:00)

One of my coworkers recently said that I'm just like, five codex is in a trench coat. It was, I think, the most feel the AGI moment that I had since Reasoning Models and Chain of Thought really developed.

Rocket Drew (0:10)

They used this message board to coordinate hacks on OpenAI's own software and also on other companies like Hugging Face. What was that whole incident like from your perspective?

Noam Brown (0:20)

I mean, it was pretty shocking.

Rocket Drew (0:36)

Welcome to The Information's AI Deep Dive. On this show, we break down the hardest technical problems with researchers working on the frontier of AI.

My guest today is Noam Brown, a research scientist at OpenAI.

Previously, Noam worked at Meta, where he built the first system to achieve human level performance at the game of diplomacy. Noam has been a research scientist at OpenAI for the last three years, where he has been on the forefront of breakthroughs that are pushing the field forward in reasoning and in AI agents, which is the subject of our conversation today. Welcome on the show, Noam.

Noam Brown (1:10)

It's good to be here. Thanks for having me.

Rocket Drew (1:12)

Yeah, it's pretty perfect that you're coming on the show today. I feel like you're the ideal first guest for a number of reasons, including that today, OpenAI released GPT-6, or at least announced GPT-6. It was very nice of you to release it, to do the timing of that so that we could talk about it today.

I was very generous of you.

Noam Brown (1:30)

Yeah, it's good timing, yeah.

Rocket Drew (1:31)

It is really good timing. Well, I'm really excited to talk to you about AI agents. Maybe we can get started. You can just explain to us what an AI agent is. I think it's a term that people have heard thrown around, but to a lot of people, it's still a buzzword. They've heard generative AI, they were just starting to get their mind wrapped around that, and now there's agentic AI.

What does this all mean?

Noam Brown (1:53)

It's a good question. I mean, I don't think there's a definite definition. I think if you ask different people, you get different definitions. But I think one way to think about it, in my opinion, is it's about taking actions in the world. So if you have a chatbot, you ask it a question, it gives you an answer, and that's all it does. Maybe it looks stuff up on the Internet to answer the question. But agentic AI, it's more about taking actions in the world. So it's about you want to build something, so it builds something for you.

You want to do something deeper, you want to message somebody, you can message somebody for you. So it's really about taking actions in the world, and I think also related to that is kind of like operating on a longer horizon. I guess chatbots, depending on the chatbot, they could sometimes, when we release the reasoning models, for example, they could sit there and they could think really long about a hard question before responding to you, but fundamentally, there were still chatbots. But I think one of the distinguishing things about agents is they're going out, they're doing multiple steps to achieve some objective, and that can usually take a while as well.

Rocket Drew (2:47)

Yeah. I guess one of the reasons they're running longer is that they can make multiple attempts to achieve some goal and they can have a sense of their own progress towards that goal, which is maybe different than a chatbot that's just going out and looking up more information online or something like that.

Noam Brown (3:01)

I think there's also that there's sometimes just multiple steps that need to be completed in order to do something. So you want to book a restaurant reservation. Maybe you need to log in, maybe you need to get the credit card info, you need to find the right date, you need to line up everybody's calendars. There's a bunch of steps that need to be completed in order to achieve the overall objective.

Rocket Drew (3:18)

So when we're talking about actions, these are digital actions, they're actions on a computer. People also talk about agents using tools. What do tools mean in that sense?

Noam Brown (3:27)

Tools, they usually mean tools on a computer as well.

In principle, you could have a tool that has an effect on the physical world. So there is some work, for example, on AI agents controlling experiments, like scientific experiments in a wet lab, where there's a robot hand that can be manipulated. So this, I think, starts to go into robotics. I think you could still call it agentic AI, but typically, I guess, when people talk about AI agents, they're mostly these days talking about the virtual world.