Benchmarks 201: Why Leaderboards > Arenas >> LLM-as-Judge

**Alessio** (0:03)
Hey, everyone, welcome to the Lit in Space podcast. This is Alessio, partner and CTO and resident of Decibel Partners, and I'm joined by my co-host, Swix, founder of SmallAI.

**Swyx** (0:13)
Hey, and today we have a super special guest that we've been trying to book on this kind of schedule for a while. It's Clémentine Fourrier.
I'm trying my best to do the French, but maybe you can do a better job of it than me.

**Clémentine Fourrier** (0:26)
This was perfect, it's Clémentine Fourrier, but your pronunciation was really on point.

**Swyx** (0:31)
There was a Fourrier, which is very sort of French intonation, which I don't really understand.
So I'll introduce you off of your LinkedIn and I would love for you to fill in the blanks. You are currently a research scientist at HuggingFace and the maintainer of the OpenLM leaderboard, which we'll talk about very shortly. Obviously you were at INRIA as well, but then you also, it looks like you also concurrently got your PhD at the same time. How does that work? Is that a very common thing?

**Clémentine Fourrier** (1:00)
So I basically did my PhD at INRIA, technically.
So INRIA funded my PhD and PhDs in France are three years, but I also worked as an engineer at INRIA before my PhD. Hence, maybe it's a confusion.

**Swyx** (1:14)
I think there's a rise in universities having sort of industrial attachments to these things. And I think it actually makes for a much more grounded study, especially if you're doing your graduate studies and all these things. I think it's rising in North America as well with Berkeley and with Waterloo and Toronto. Cool. There's a lot of other things we can introduce. I can't really pronounce the name of the university he went to.
What else should people know?

**Clémentine Fourrier** (1:44)
So I actually technically am an engineer in geology. I studied rocks and I graduated in 2015 after having done extensive studies about rocks and I discovered I was very bad at it, but I was very good at computer science, so I went to computer science. What stuck with me though is that geology is very much an experimental science and I think that machine learning is very much an experimental science too, even though people want to claim that it's pure math.
I worked on several machine learning projects throughout the years, a bit of the prediction of illnesses in the brain. At Brain and Spine Institute in Paris, I worked as an engineer in a research team in NLP where I did my thesis and then I joined HuggingFace.

**Swyx** (2:32)
Do you have a favorite rock fact or rock story before we get into the NLP stuff?

**Clémentine Fourrier** (2:39)
I was not expecting this question.

**Swyx** (2:44)
I did my geography A-levels and I always loved learning about isostasy and stuff like that, where you have different plates kind of up and down in the mantle. I don't think people think about vertical dimensions to geographical plates, but it's real.

**Clémentine Fourrier** (3:02)
Yeah, definitely. And when you do geology, the time scale is just not the same. There is one specific place in France where you can see rocks that are one billion years old. And the sheer scale of this is huge.
Yeah, that's what I loved about geology, that the scale is completely different and it makes us see the rest of the world in perspective, I guess. We are like a blink in the length of time of the earth.

**Swyx** (3:31)
But a very significant blink, so you went from large monoliths to large language models. I don't know how to make that transition there.
Could you describe your journey into HuggingFace? Obviously, I think you're like our second or third person from HuggingFace on the podcast and it's like the definitional sort of OpenAI company, maybe the real OpenAI.

**Clémentine Fourrier** (3:57)
Yeah, I did.
So at the end of my PhD, I realized that I did not want to stay in academia. And I actually got contacted by Meta because they wanted to offer me an internship. And I was like, wow, I can do an internship during my PhD. Where do I want to do an internship? And so I applied to HuggingFace. Thank you Meta for opening this door for me. And I actually was hired to work on pre-trained graph transformers. Can we train foundational graph transformer models?
And it was a very interesting project, but it was a bit hard to accomplish with the resources we had at the time. We tried it for three months, gave it three months more. So the first three months were my internship, three months more were my first three months at HuggingFace, and then we dropped it.