🔬Searching the Space of All Possible Materials — Prof. Max Welling, CuspAI

**Max Welling** (0:00)
I want to think of it as what I would call a sort of a physics processing unit, like a PPU, right? Which is you have digital processing units, and then you have physics processing units. So it's basically nature doing computations for you. It's the fastest computer known, if possible even. It's a bit hard to program, because you have to do all these experiments. Those are quite bulky, it's like a very large sort of thing you have to do. But in a way, it is a computation, and that's the way I want to see it. You can do computations in a data center, and then you can ask nature to do some computations. Your interface with nature is a bit more complicated, but then these things will have to seamlessly work together to get to a new material that you're interested in.

**Brandon** (0:45)
Yeah, it's a pleasure to have Max Woehling as a guest today. Max has done so much over his career that I've been so excited about. If you're in the deep learning community, you probably know Max for his work on variational autocoders, which has literally stood the test of time or officially stood the test of time. If you are a scientist, you probably know him for his like, pioneering work on graph neural networks, on aqua variants. And if you're a material science, you probably know about his new startup, CuspAI. Max has a long history doing lots of cool problems. You started in quantum gravity, which is, I think, very different than all of these other things you worked on. The first question for AI engineers and for scientists, what is the thread in how you think about problems? What is the thread in the type of things which excite you? And how do you decide what is the next big thing you want to work on?

**Max Welling** (1:35)
So it has actually evolved a lot. In my young days, let's put it, I would just follow what I would find super interesting. I have kind of this sensor. I think many people have, but maybe not really sort of use very much, which is like you get this feeling about getting very excited about some problem, right? Like it could be what's inside of a black hole, or what's at the boundary of the universe, or what is quantum mechanics actually all about. And so I followed that basically throughout my career. But I have to say that as you get older, this changes a little bit in the sense that there's a new dimension coming to it, and this is impact. Working in two-dimensional quantum gravity, you pretty much guarantee there's going to be no impact in what you do, relative to maybe a few papers, but not in this world, at this energy scale. As I get closer to retirement, which is fortunately still 10 years away or so, I do want to make a positive impact in the world. And I got pretty worried about climate change.
And I think we should, you know, and politics seems to have a hard time solving it, especially these days. And so I thought better work on it from the technology side. And that's why we started CuspAI. But there's also a lot of really interesting science problems in, you know, material science. And so it's kind of combining both the impact you can make with it, as well as the interesting science. So it's sort of these two dimensions, like working on things which you feel is like, oh, there's something very deep going on here. And on the other hand, trying to build tools that can actually make a real impact in the world.

**RJ** (3:20)
So the thread that, when I look back, look at the different things you worked out, some of them seem reconnected, like the physics to to Equivariance and Grapherial networks, maybe. And that seems to be somewhat related to Casp. Do you have a thread through there?

**Max Welling** (3:41)
Yeah, I think physics is the thread. So having done, you know, spent a lot of time in theoretical physics, I think there is first very fundamental and exciting questions, things that haven't actually been figured out in quantum gravity. So there's really the frontier. There's also a lot of mathematical tools that you can use, right? In, for instance, in particle physics, but also in general relativity, sort of symmetry space play an enormously important role. And this goes all the way to gauge symmetries as well. And so applying these kinds of symmetries to machine learning was actually, you know, I thought of it as a very deep and interesting mathematical problem. I did this with Taco Cohen, and Taco Cohen was the main driver behind this. He went all the way from just simple rotational symmetries all the way to gauge symmetries on spheres and stuff like that. So Maurice Weiler, who was also here when he was a PhD student with me, he wrote an entire book which I can really recommend about the role of symmetries in AI and machine learning. I find this a very deep and interesting problem. So more recently, I've taken a sort of different path, which is the relationship between diffusion models and a field called stochastic thermodynamics. This is basically the thermodynamics, which is a theory of equilibrium, but then formulated for out-of-equilibrium systems. It turns out that the mathematics that we use for diffusion models, but even for reinforcement learning, for Schrodinger bridges, for MCMC sampling has the same mathematics as this physical theory of non-equilibrium systems.