The Physics Reason AI Works When It Shouldn’t

**Nigel Goldenfeld** (0:00)
And you're way, way into that regime where you're just fitting noise and the whole thing shouldn't work. It obviously shouldn't, and yet it does. Francis Crick said, Look, there's no way that life got to this level of complexity in such a short period of time. It must have come from outer space. The genetic code is optimal in the sense of minimizing errors. If you want to know what is the purpose of life, the purpose of life is to help planets come into equilibrium.
So a phase transition you know is like, for example, what happens to a piece of metal when I cool it below a certain temperature and I want to know, can I use it to stick the pictures of my kids' holiday pictures on the door of my refrigerator? And the answer is yes, if it's magnetic, it'll stick with a piece of metal holding the picture up. And if it's not magnetic, it won't stick. And if you take that piece of metal that works as a magnet, fridge magnet, and you heat it up, it will stop becoming a magnet. And that's called a phase transition, as you know, and maybe some of your listeners know or viewers know. The interesting question you might ask is, as I get closer and closer to the temperature where the magnetization disappears, how does the magnetization disappear? Does it disappear gradually, in fact? And if you ask how much magnetization there is, the answer is it goes like the square root of the difference between the temperature you're at and the critical temperature where the magnetization fully goes to zero. At least that's what you would expect. And that's what very generic, very persuasive, simple theory, theoretical arguments that anybody can understand. I can explain it to my class in literally 20 seconds. That's what you would predict. When you do the experiment, you find that it doesn't go like the square root of the critical temperature minus the temperature. It goes like the Tc minus T to a power like 0.3265136, some weird, weird number like that. And you might say, well, it's just a more accurate number. The problem is that there's no known way, or there was no known way to account for the fact that the member is not a half.
It's like to prove it's a half, all I need to know is that magnets can be either magnetized north or magnetized south. And that's basically it. It's an argument that is so compelling, it can't possibly be wrong. And yet in the decades from the 1940s up to the middle 1960s, it was discovered that it was wrong. And it wasn't just only the magnetization, there were other thermodynamic properties like heat capacity and things like this, which I won't go into, which also have a similar unaccountable behavior. And there was a fact that these numbers, themselves, not particularly important. It was the fact that you could even explain in principle why they are not these simple numbers like a half and so on. And that was the reason for the puzzle.
And the explanation is a truly mind-boggling explanation. But just to tell you the outline of the story, this phenomenon was addressed by Leo Kadanoff, Ben Widom, eventually Ken Wilson.
And they invented this process of looking at a physical system on different scales of energy. So you could look at matter at the scale of this room, you could hear the sound waves, you can see light bouncing off the surfaces. On the other hand, if you want to go and see that there are atoms and see that there are quarks and things like this, you need to build a machine that's put in a tunnel 70 miles long under the Swiss Alps in order to be able to see things like that. So what you can see depends on what energy you look at it at and what time scale and what length scale you look at it at. And the same thing turns out to be true of the laws of physics themselves.
And Leo Kadanoff was the first person who realized that. And Ken Wilson turned it into a mathematical tool, which was called the renormalization group. In fact, it's not even a group. You ask whether it's a group. It's actually a semi-group. And the idea was this. Take a physical system and then just say, well, I've got magnetic dipole moments. They're really spins of electrons. But we'll just call them magnetic dipole moments. They're in this bit of the sample. We'll just lump them together into one effective dipole moment. Because in this patch over here, 80% of them are pointing up. 20% are pointing down. So we'll just say, OK, it's basically just a spin pointing up. And so you sort of block things up in that way. And then once you've done that once, you can do it again, and again, and again.