#76 - Carola Schönlieb

**Craig Cannon** (0:00)
Hey, how's it going? This is Craig Cannon, and you're listening to Y Combinator's podcast. Today's episode is with Carola Schönlieb. Carola is an applied mathematician at the University of Cambridge. She's also a Turing Fellow at the Alan Turing Institute, and the head of the Image Analysis Group at Cambridge's Department of Applied Mathematics and Theoretical Physics.
In this episode, we cover mathematical approaches to image processing. All right, here we go.
We ought to start with a little bit of your background. So, what did you start researching, and then what are you researching now?

**Carola Schönlieb** (0:34)
So I started out my research in mathematics in Austria, in Vienna, where I actually didn't look at image processing or imaging at all. I started out with so-called partial differential equations, which are equations of a function and its derivatives. So you take an express change over time or space, and they are models for various natural phenomena in physics and biology. Lots of things are explained by these differential equations. And my first paper, again, had nothing to do with image processing. It was actually on the Cunhillet equation, which is an equation that describes phase separation and coarsening in alloys, in metallic alloys, for instance. So when you cool them down to a certain temperature, you have a mixture of two. And if you cool them down to a certain temperature, they are starting to separate from each other and coarsen out and build these larger areas. And so there is an equation that models this kind of phenomenon, which is the Cunhillet equation. And my first paper was on the stability analysis of a certain type of solutions to this Cunhillet equation.
Stability analysis, meaning that if you perturb your initial condition a little bit, how much is your stationary solution that is when you let time evolve infinitely?
How, you know, when a stationary state is a state of where the system is in no change anymore, how much do these stationary states differ from each other when you just perturb the initial condition a little bit?

**Craig Cannon** (2:11)
And this is in the context of creating alloys or building structure alloys for structures, or what was their new particular purpose?

**Carola Schönlieb** (2:17)
Well, the purpose is a lot with these differential equations to simulate certain phenomena. And so if you understand how stable these stationary states are, so if you are at a stationary state and then you perturb the stationary state a little bit, is it going back to the same stationary state? Or is it going somewhere completely different? So you kind of understand how these systems react to perturbations that are naturally occurring because we are in real life and things happen.
Yeah, so it's more an understanding of the physical processes involved in, you know, mixture of alloys, for instance, or things like that.

**Craig Cannon** (2:57)
And were you at a technical university where you would be like focusing on alloys or this was a personal interest?

**Carola Schönlieb** (3:02)
Not at all. So it was just, so actually, you know, a lot of applied mathematics on the continent, which is everything else in the UK, basically here in Europe is applied mathematics very much means that what you're doing is inspired by applications, but eventually you end up with a mathematical problem. So it was really the driving factor was, well, we were interested in analyzing this equation and there were techniques coming up that are kind of cool. Yeah, so it was just a kind of intellectual interest in this equation. That was the driving factor for this particular paper.
But then during writing this paper, research at UCLA, researchers at UCLA, in particular the group of Andrea Bertozzi, used this same equation to do image restoration.
And image restoration meaning you have a digital image and there are parts of this image which are damaged for some reason or where you have objects which are occluding some other object of interest that you want to get rid of the occlusion or something like this. So you have one part in the image that you somehow want to replace by something that is suggested by the surrounding area of this region.

**Craig Cannon** (4:29)
So is this similar to like content aware fill in Photoshop? Exactly, exactly.

**Carola Schönlieb** (4:38)
It actually does. And I mean, also the content aware fill is actually very much based on some of the things that have been initiated by people like Andrea Batozzi. So I mean, the technique is different in what Photoshop is using, but it's still based on research in mathematics. In fact, it's a differential equation. Maybe if you wish that it's not the Carnier-Litt equation, but it's a different type of differential equation that is non-local, is taking patches in images and kind of copy and pasting them into the region that you want to replace.