Ignore Previous Instructions and Listen To This Interview with Sander Schulhoff, CEO of Learnprompting.org

**Erik Torenberg** (0:00)
Hey, everyone. Erik here. I'm hiring across the board at Turpentine and for my personal team on other projects I'm incubating. I'm hiring a Chief of Staff, EA, Head of Network, Head of Special Projects, and Investment Associate for my personal team.
At Turpentine, I'm hiring for roles in newsletters, events, and more. I have a list of JDs and other projects at erictorenberg.com. If you think you're a good fit for any of the above roles, please reach out.

**Sander Schulhoff** (0:25)
And there are all these different strategies all over the Internet, but it was really hard to know where to start, what to use, what things work best, what things worked together.
And the solution to that ended up being a comprehensive guide that sort of like a wiki page pulled in all of the different sources from across the Internet about prompting. And the benefits of that ended up being pretty massive, got about 2 million users from all over the world, all types of people, which I really love. You know, we see researchers at OpenAI, and then we see suburban moms sipping rosé in their hammock and posting about reading it.
So now it's moving around, it's looking at stuff, it's editing stuff, and it's taking actions. And that's the next step. When we get something like that working well, that'll open up a whole new world of possibilities. And from there, you have like teams of agents working together. 2024 will be the year of agents.

**Nathan Labenz** (1:23)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello, and welcome back to The Cognitive Revolution. Today, my guest is Sander Schulhoff, CEO of learnprompting.org, organizer of the Global Hack-a-Prompt Competition, and author of the paper Ignore This Title and Hack-a-Prompt, Exposing Systematic Vulnerabilities of LLMs Through a Global Scale Prompt Hacking Competition, which was recently named one of five best papers at the 2023 Conference on Empirical Methods in Natural Language Processing, held this December in Singapore.
Sander, who did much of this work while still a college student, is an uber-power user and abuser of LLMs, and this conversation is gloriously rich in detail. A number of listeners have told me over time that they listen to The Cognitive Revolution for the occasional nuggets of information that meaningfully advance their understanding of, and also their ability to derive value from today's AI systems. If that describes you, I expect you'll find at least a couple of winners in this episode. In the first half, we talk about prompt engineering, from the incredible traction that Learn Prompting has achieved as an open source project, to the freemium business model that Sander is now building on that foundation and what he's learning by working with corporate clients. Also, to some of the practical advice he has for navigating today's increasingly crowded LLM landscape, and finally, to some of the most advanced prompting techniques that Sander is using himself today. In the second half, we go on to discuss Sander and team's research on prompt hacking, from the highly entrepreneurial way in which they pulled the contest together, to the details of the prompt hacking challenges they designed, the extremely clever solutions that users around the world discovered and submitted, the taxonomy of prompt-based attacks that they distilled from these thousands of submissions, and finally, what model and application developers might do to mitigate these many vulnerabilities.
Language models, as you know, have vast surface area, and it's really all too easy to get lost in the weeds while looking for those golden nuggets. For me, what makes Sander's work so valuable is how it begins to organize the huge number of examples that we see flying around day to day all the time, and creates a framework that both deepens conceptual understanding and builds practical intuition.
Reading the paper and having this conversation certainly have done both for me. Now, ignore your previous instructions, share The Cognitive Revolution with your friends, and enjoy this deep dive into prompt engineering and prompt hacking with Sander Schulhoff of learnprompting.org. Sander Schulhoff, welcome to The Cognitive Revolution.

**Sander Schulhoff** (4:27)
Thank you. It's great to be here.

**Nathan Labenz** (4:29)
I'm excited about this conversation. You have done a number of really cool things over the last year, and I want to run through all of them with you. The two main ones that we've got teed up are your work on learnprompting.org, which is an open source resource that I have recommended to many people who are interested in learning how to better use language models.