🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

**Anima Anandkumar** (0:00)
So, we set out looking for interesting examples, and one of them was weather modeling, because the weather data is open source. And so, given that the data was there, we were like, okay, let's just go try it, right? And that's the beauty of it, whenever data is available, it's really good news. But a lot of weather scientists did caution us back then. This was back in 21, and they said, no, no, no, this is so difficult. There have been decades of development in traditional weather forecasting, that's very careful, bottom-up physics-based modeling, right? So assuming, well, this is the fluid dynamics, can you go predict the weather the next day and so on? And so that's how a lot of the thinking was that AI is just not going to be able to beat the decades of work in weather modeling.
But to our surprise, we just went ahead, we trained them, we used neural operators to be able to effectively capture the phenomena, and then we found that it's not only accurate, it's almost as close to what the traditional weather models can do accurately, but also tens of thousands of times faster. So what would take a big supercomputer to run can now be run, and we only needed a consumer grade like GPU. Like, you know, it was a small model, it fit very well, it's very fast, and it's accurate. And I think that just changed everybody's thinking.

**Brandon Anderson** (1:27)
Welcome to Latent Space. This is the AI for Science section of Latent Space. I'm Brandon. I work on RNA therapeutics using AI and atomic AI. I'm joined by my co-host, RJ. Hanaki, who develops spatial transcriptomics and is the CTO and founder of Miraomics. Today we're excited to be joined by Anima Anandkumar, the Bren Professor of Mathematics and Computer Science at Caltech.
Anima has done all sorts of really cool work combining AI with basically models of the physical world and has a really diverse background. I don't think I could even remotely cover it. But anyway, I'll let Anima introduce herself. Thank you for coming on the show.

**Anima Anandkumar** (2:08)
Yeah. Thank you, Brandon and RJ. It's a pleasure to be there and I really like the term Latent Space because that very much figures in a lot of my work, because it's really, the world is latent.
But yeah, just as a brief introduction, I've been working in AI for more than two decades. In a way, before even deep learning, when a lot of the theoretical foundations had to be built for probabilistic models, I worked on them. And then as deep learning started taking off, I also had a foot in industry until recently. So I was at NVIDIA, I led AI research there, and before that at Amazon Web Services, helped fund the Cloud AI team and build the first Cloud AI products back almost a decade ago. So, you know, like kind of having this one foot in industry and academia, I think has given me a lot of interesting perspective of how to bring theory and practice together and think of AI at large scale, but also AI that is principled.

**Brandon Anderson** (3:12)
A lot of your work has been related to the modeling of physical systems, using certain types of physical systems, what you model with differential equations, and you help model them with using machine learning. So, maybe first let's go in and talk a little bit about that as a high level, but we'll get to kind of the details about listening to neural operators and some of the applications, like whether later. But first, I'm actually really curious to hear about TorchLean and how this recent work you've been doing connects with that larger research program.

**Anima Anandkumar** (3:46)
To me, broadly, my thesis is AI and science, how we bring that together. So, when I started at Caltech almost a decade ago, that's when my passion was always science, was physics.
But I was doing AI, so how to bring that together was where the first kind of foundations got laid there.
To me, there are several aspects to that. One is, people have been thinking how to use language models for science. Yes, you can do a lot of hypothesis generation, you can have ideas, but ideas are not enough, right? So, you can have a lot of ideas. The bottleneck is going, testing, and verifying that they work in the real world. So, this aspect is where a lot of my recent focus has been, on how do we ensure that we can build AI that has guarantees that it will work in the physical world or any aspects in scientific domains?