“Engineers are becoming sorcerers” | The future of software development with OpenAI’s Sherwin Wu

**Sherwin Wu** (0:00)
95% of engineers use Codex. 100% of our PRs are reviewed by Codex.

**Lenny Rachitsky** (0:04)
For engineers, I don't know what job has changed more in the past couple of years.

**Sherwin Wu** (0:09)
Engineers are becoming tech leads. They're managing fleets and fleets of agents. It literally feels like we're wizards casting all these spells, and these spells are kind of like going out and doing things for you.

**Lenny Rachitsky** (0:17)
What do you think people aren't pricing in yet?

**Sherwin Wu** (0:19)
The second or third order effects of the one-person billion dollar startup. To enable a one-person billion dollar startup, there might be a hundred other small startups building bespoke software. So I think we might actually enter into a golden age of B2B SaaS.

**Lenny Rachitsky** (0:30)
I've been hearing more and more there's this stress people feel when their agents aren't working.

**Sherwin Wu** (0:34)
There's a team that's actually doing an experiment right now with an OpenAI where they are maintaining a 100% Codex written code base. They run into the exact problems that you're describing. And so usually you're like, all right, I'll roll up my sleeves and figure it out. This team doesn't have that escape hatch.

**Lenny Rachitsky** (0:47)
You've shared that listening to customers is not always the right strategy in AI.

**Sherwin Wu** (0:50)
The field and the models themselves are just changing so, so quickly. They tend to disrupt themselves. The models will eat your scaffolding for breakfast.

**Lenny Rachitsky** (0:59)
What's your advice to folks that are like, okay, I don't want to miss the boat?

**Sherwin Wu** (1:02)
Make sure you're building for where the models are going and not where they are today. There's a quote from Kevin Weil, our VP of Science here. He likes saying this is the worst the models will ever be.

**Lenny Rachitsky** (1:11)
Today, my guest is Sherwin Wu, Head of Engineering for OpenAI's API and Developer Platform. Considering that essentially every AI startup integrates with OpenAI's APIs, Sherwin has an incredibly unique and broad view into what is going on and where things are heading. Let's get into it after a short word from our wonderful sponsors. Today's episode is brought to you by DX, the Developer Intelligence Platform designed by leading researchers. To thrive in the AI era, organizations need to adapt quickly. But many organization leaders struggle to answer pressing questions like, which tools are working? How are they being used? What's actually driving value? DX provides the data and insights that leaders need to navigate this shift. With DX, companies like Dropbox, booking.com, Adyen, and Intercom get a deep understanding of how AI is providing value to their developers and what impact AI is having on engineering productivity. To learn more, visit DX's website at getdx.com/lenny. That's getdx.com/lenny. Applications break in all kinds of ways. Crashes, slowdowns, regressions, and the stuff that you only see once real users show up. Sentry catches it all. See what happened, where and why. Down to the commit that introduced the error, the developer who shipped it, and the exact line of code all in one connected view. I've definitely tried the five tabs and slack thread approach to debugging. This is better. Sentry shows you how the request moved, what ran, what slowed down, and what users saw. Seer, Sentry's AI debugging agent, takes it from there. It uses all of that Sentry context to tell you the root cause, suggest a fix, and even opens a PR for you. It also reviews your PRs and flags any breaking changes with fixes ready to go. Try Sentry and Seer for free at sentry.io/lenny, and use code Lenny for $100 in Sentry credits. That's S-E-N-T-R-Y dot I-O slash Lenny.
Sherwin, thank you so much for being here, and welcome to the podcast.

**Sherwin Wu** (3:20)
Thank you. Thank you for having me.

**Lenny Rachitsky** (3:21)
I want to start with what's feeling like a barometer of progress in AI, especially in engineering. What percentage of your code, if you even write code anymore, and your team's code is written by AI at this point?

**Sherwin Wu** (3:33)
I do write code occasionally now still. I'd actually say for managers like myself, it's way easier to use these AI tools than to manually code at this point. So I know for myself and some of the other EMs, engineering managers at OpenAI, all of our code is written by Codex at this point. But more broadly, there's just so much energy. There's a tangible energy internally around just how far these tools have gotten, how good Codex as a tool has gotten for us. And it's a little hard for us to exactly measure how much of the code is written because the vast majority of it, I'd say close to 100% is usually generated by AI first. What we do track though is at this point, the vast majority of engineers use Codex on a daily basis. So 95% of engineers use Codex. 100% of our PRs are reviewed by Codex daily as well. So basically any code that goes into production that's merged in, Codex has its eyes on and suggests improvements, suggests changes in the PRs. And so that's what we're seeing internally. But by and large, the most exciting is just the energy that there is. Another observation that we've had is engineers who tend to use Codex more open way more PRs. So they're actually opening 70% more PRs than the engineers who aren't using Codex as much. And the gap is widening. So I feel like the people who are opening more PRs are starting to learn how to use the tool more and more, get more efficient, and that 70% gap keeps growing over time. And so it might have actually increased since I last looked at the number.