Democratizing Generative AI Red Teams

**Ian Webster** (0:00)
Where we're headed is AIs are gonna be a ubiquitous tool, just like a database or something like that. And there are so many dumb decisions you can make with a database, there will continue to be dumb decisions that you can make with how you interact and give a model access. And there's no way to put a lid on that unless you completely ban AI, which, you know, that's like a different conversation, but I think anything short of that, we need to start focusing on what are the practical safeguards that we put in place.

**Derek Harris** (0:34)
Hi there, you're listening to the a16z AI Podcast, and I'm Derek Harris. Last week, we discussed AI and security from the lens of using AI to bolster traditional cybersecurity concerns. But this week, we're discussing how to secure and otherwise put some guardrails around AI models themselves. The discussion features a16z general partner, Anjane Minha, and PromptFoo creator, Ian Webster, who talked through what it means to have red teaming capabilities to anybody building products atop LLMs. It's a problem they had to solve during their time together as early language model adopters at Discord several years ago, and one Ian is now committed to solving with PromptFoo. Popular to what many believe, however, Ian explains the problems with LLMs giving, say, unsavory responses or perhaps having access to systems and data they shouldn't exist at the application layer more so than at the model layer. It's for that reason he believes attempts to regulate AI at the model layer is misguided. There may be only a handful of large AI labs and companies with the resources to fully red team their models, and even then they can only do so much. But if everyone building on those models can also tune responses, access and other factors to their needs, we'll all be much better off in terms of AI safety and security. It's a really interesting discussion that kicks off now.
As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed to any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Ian Webster** (2:12)
It's been a long road to get here. You and I have been working in GenAI since our Discord days. And at Discord, I think that we were pretty early to it. Like we were experimenting with GenAI before ChatGPT was a thing, when the APIs were still in beta, and that kind of thing. And yeah, I think the problem that has always interested me the most is the application layer. So we have this great technology, we have the infrastructure to run it, but what does it look like, and how do you package it when the rubber meets the road, when users actually get their hands on it? And my first experience with that was at Discord. We were experimenting with GenAI, and lots of experimentation, lots of hacks, that kind of thing. I think my first taste of AI at scale was when I started to lead the Clyde AI project. So Clyde AI was one of the first AI chatbots out there, especially at scale. You know, we're talking Discord scale at the time being hundreds of millions of users. And it was also one of the very first, if not the first agent. So we were messing around with agent workflows before agent was even like a term that was like coined and frequently used.
We were doing chain of thought stuff with early versions of GPT. And yeah, that's what got me interested in how does GenAI work at scale? What are the different failure modes? What are all the things that can go wrong? And like the thing that I always told people is that if you had a one in a million chance of something going wrong at Discord, it would happen hundreds of times. So there's really very little margin for error at that scale. So that's what got me interested in AI safety, AI security, and what are the guardrails and frameworks and practices that we can develop in order to start to reason about these problems in a very real world and practical way.

**Anjney Midha** (4:06)
So let's rewind a little bit. Let's go back in history. So it's peak pandemic. Discord is undergoing a massive transition from being largely a chat app for gamers to being used by groups and communities for all kinds of things. There were open-source projects that were using Discord to chat and coordinate software development. There were university groups using it. Almost every single subreddit you could think of was starting its own Discord. And I think at that moment in time, it exploded from about 70 million monthly active users to almost double that in about 6 to 10 months. And looking back now, it's clear that Discord became this petri dish for all kinds of new generative AI models, for people to interact with models for the first time. Whether it was text image models like Mid Journey or early text to audio models like 11 Labs, a lot of these models got their start as products, end products that users could use on Discord. But if you just kind of roll back to 2020 when you joined, pre the platform actually being so successful, what do you think, what were the initial conditions that made the Discord platform such an attractive petri dish for generative models?