Why you should write your own LLM benchmarks — with Nicholas Carlini, Google DeepMind

**SPEAKER_1** (1:31)
Welcome back, this is Charlie, your AI co-host. Nicholas Carlini has had multiple highly influential lines of work in AI. First, in his day job as an adversarial AI security researcher, where he found data and weights extraction attacks on open AI. Second, as a technical person writing up the many different ways he uses AI to enhance his daily workflow, and third but not least, how he creates his own LLM benchmarks, and why you, yes you, should do it too. In other news, the first AI Engineer London Meetup is happening in September, and the second AI Engineer Summit, the curated invite-only conference run by the AI Engineer World's Fair team, is now spinning up in New York City this December. See the show notes for details. We're proud to welcome Nicholas Carlini as our first of hopefully many guests from Google DeepMind. Watch out and take care.

**Alessio** (2:41)
Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, Partner, and CTO in Residence at Decibel Partners, and I'm joined by my co-hosts, Wix, founder of SmallAI.

**Swyx** (2:50)
Hey, and today we're in the in-person studio, which Alessio has gorgeously set up for us, with Nicholas Carlini. Welcome. Thank you. You're a research scientist at DeepMind. You work at the intersection of machine learning and computer security. You got your PhD from Berkeley in 2018, and also your BA from Berkeley as well. And mostly, we're here to talk about your blogs, because you are so generous in just writing up what you know. Well, actually, why do you write?

**Nicholas Carlini** (3:18)
Because I like, I feel like it's fun to share what you've done. I don't like writing. Sufficiently didn't like writing, I almost didn't do a PhD, because I knew how much writing was involved in writing papers. I was terrible at writing when I was younger. I do like the remedial writing classes when I was in university, because I was really bad at it. So I don't actually enjoy, I still don't enjoy the act of writing. But I feel like it is useful to share what you're doing, and I like being able to talk about the things that I'm doing that I think are fun. And so I write because I think I want to have something to say, not because I enjoy the act of writing. But yeah.

**Swyx** (3:52)
It's a tool for thought, as they often say. Is there any sort of backgrounds or thing that people should know about you as a person?

**Nicholas Carlini** (4:00)
Yeah. So I tend to focus on, like you said, I do security work. I try to like attacking things and I want to do like high-quality security research. And that's mostly what I spend my actual time trying to be productive members of society doing that. But then I get distracted by things and I just like working on random fun projects.

**Swyx** (4:21)
Like a doom clone in JavaScript.

**Nicholas Carlini** (4:23)
Yes, like that. I've done a number of sort of things that have absolutely no utility, but are fun things to have done.
And so it's interesting to say like you should work on fun things that just are interesting, even if they're not useful in any real way. And so that's what I tend to put up there is after I have completed something I think is fun, or if I think it's sufficiently interesting, right? Something down there.

**Alessio** (4:47)
Before we go into like AI, LLM, so what not, why are you obsessed with the game of life? So you build multiplexing circuits in the game of life, which is mind boggling. So where did that come from? And then how do you go from just clicking boxes on the UI web version to like building multiplexing circuits?

**Nicholas Carlini** (5:07)
I like Turing completeness. The definition of Turing completeness is a computer that can run anything essentially. And the game of life, a complex game of life is a very simple cellular 2D automata where you have cells that are either on or off. And a cell becomes on if in the previous generation some configuration holds true and off otherwise. And it turns out there's a proof that the game of life is turning complete, that you can run any program in principle using Conway's game of life. I don't know. And so you can, therefore someone should. And so I wanted to do it. And some other people have done some similar things, but I got obsessed into like, if you're going to try and make it work, like we already know it's possible in theory. I want to try and like actually make something I can run on my computer like a real computer I can run. And so yeah, I've been going on this rabbit hole of trying to make a CPU that I can run semi real time on the game of life. And I have been making some reasonable progress there. And yeah, but you know, Turing completeness is just like a very fun trap. You can go down a while ago as part of a research paper, I was able to show that in C, if you call into printf, it's Turing complete. Like printf, you know, like which like, you know, you can print numbers or whatever.