Narrow Superintelligence: Why AI Strengths May Not Transfer

Nate B. Jones (0:00)

OpenAI just released more than 700 math research papers, and the part that's killing me is how they say they did it. Almost every result came from a single prompt using a model they haven't even released. It's the clearest example yet of LLM-driven superintelligence. You've been hearing about it for years now, right? You've been hearing about it on SNL from Anderson Cooper. It's coming for your job. The world is over. And I get it. This looks like proof, doesn't it? Well, I read the release and I came away thinking the opposite. I still know engineers who will not let a coding agent near production. In fact, most of them don't. They put a bunch of evals in place and they have a zero trust environment. That difference should be something you pay attention to, because isn't code checkable? Every lab is pouring money into models that can code. There is easy verifiability here. Does it run or does it not? If the model is superhuman at math, you would think it would be crushing your code base by now, and it's not.

That gap is the most useful thing that you can understand about AI right now, because it tells you where your job stands and why. So today, we're going to look at why you actually ought to care about math, even if you didn't take calculus, what we learned from all of the mathematical insight this week, and what this doesn't tell us about work. Because I think we're looking at a world with narrow superintelligence and some of the consequences of narrow superintelligence are very surprising. We're going to follow that into code, we're going to follow that into business work, and then we're going to talk about your skills. What's worth learning in this world? How do we use the abilities these systems give us, and how do we respond and get better at things that we actually care about? So let's start with what these papers actually contain. We're going to get a little nerdy, but not too nerdy. On a math exam, someone already knows the answer, right? In mathematical research, you're trying to establish something that people haven't actually figured out yet. So a useful result becomes something that other researchers can build on. Eventually, it can change the methods available to engineers and the products that we all use every day. Let's take something super nerdy like matrix multiplication. Maybe you learned it in high school, maybe you didn't. That's the arithmetic involved in multiplying grids of numbers, and AI systems do piles of it. And one of these results improves the theoretical bound on how much math you need as these grids grow. Another nerdy result concerns learning what's inside a material from measurement taken at the surface, like measuring the surface of a beach ball and learning what's inside it. For a particular class of elastic materials, the proof basically says you can do that. You can measure the surface of the beach ball and learn what's inside it. If you're trying to inspect an object without cutting it open, that's super useful knowledge. It tells you the information you need is present in the surface measurements. And now you can work on recovering it under real world conditions where measurements are incomplete and noisy. And there is reason to care about this, even if math is like your least favorite subject. The discoveries are going to reach you eventually through cheaper computing, through better ways of inspecting things, through improved software. And so a model might improve the tools available in your industry with discoveries like this math moment. But that doesn't mean it's an expert at running a business in your industry, right? In other words, the discoveries from the math papers today may get all over the world. But that doesn't mean that, magically speaking, a model that can solve these math papers is going to transfer its judgment everywhere we work day to day. But one of the things that I want to say really clearly is that even though this feels a long way away from day to day, it may get there surprisingly fast precisely because we have AI. In other words, AI may make it possible for us to pull this theory into engineering, into the kinds of technology that we can use day to day much faster than we would have 10 years ago.

And so that's part of why I'm excited. And I think that it gives us a chance to actually improve upon our world in probably hundreds of ways, all in one day. And the model that produced all this, it's still unreleased. I don't have access to it. You don't have access to it. What researchers have in their hands is just the work this model produced. And what OpenAI reports is that this model is so powerful that any given result that I just described only took about three hours of thinking. So the model did something in three hours with some of these problems that mathematicians can't do their whole lives. That's super intelligence. And there's no other way to describe it. And so what can we learn here? Why do we care? Let's set aside the math piece. Let's pretend we don't care for a second. What can we learn about AI and ourselves? I think narrow super intelligence is the right way to understand what we're seeing. It's a system that can exceed human abilities in very particular kinds of intellectual work, while the rest of the abilities develop very unevenly. And that makes the math piece especially interesting because you can state exactly what success would mean, and then spend your entire life trying to achieve it. And the question can be perfectly clear along the way, while the reasoning required to get there is extraordinarily difficult, convoluted, takes mastering multiple branches of mathematics at once. When you train a system by rewarding correct answers, being able to recognize a correct answer helps a lot. And math gives you that possibility alongside problems that get extremely hard. And my suspicion is that it is unusually well-formulated for the kinds of super-intelligent reasoning gains that we are seeing right now. OpenAI says it expanded into open research problems because existing mathematical evaluations were saturated. The tests just weren't hard enough anymore. The lab needed harder questions, and those questions were things that people had built their careers around and had not yet solved. And that helps explain the collision with the mathematics community. I don't think you need to imagine a world where OpenAI is setting out to hurt mathematicians. We have no evidence of that. The researchers at the lab needed to find out how capable their model had become. They had to reach for difficult problems, and to somebody else, that problem represents years of work and the contribution they hope to make. And to the lab, they're just looking to find something hard enough to test the model on. But first, there's a big assumption here that I think we need to examine about our own work. A breakthrough in mathematical reasoning doesn't tell us very much about how the same capability transfers to other areas of work, like engineering judgment or business intuition.