Extreme Harness Engineering for Token Billionaires: 1M LOC, 1B toks/day, 0% human code, 0% human review — Ryan Lopopolo, OpenAI Frontier & Symphony

**Ryan Lopopolo** (0:00)
I do think that there is an interesting space to explore here with Codex, the harness, as part of building AI products, right? There's a ton of momentum around getting the models to be good at coding. We've seen big leaps in, like, the task complexity with each incremental model release, where if you can figure out how to collapse a product that you're trying to build, a user journey that you're trying to solve into code, it's pretty natural to use the Codex Harness to solve that problem for you. It's done all the wiring and lets you just communicate in prompts to let the model cook. You have to step back, right? Like, you need to take a systems thinking mindset to things and constantly be asking, where is the agent making mistakes? Where am I spending my time? How can I not spend that time going forward and then build confidence in the automation that I'm putting in place so I have sold this part of the STLC?

**swyx** (1:02)
All right, we're in the studio with Ryan Lopopolo from OpenAI. Welcome.

**Ryan Lopopolo** (1:06)
Hi.

**swyx** (1:07)
Thanks for visiting San Francisco and thanks for spending some time with us.

**Ryan Lopopolo** (1:09)
Yeah, thank you. I'm super excited to be here.

**swyx** (1:11)
You wrote a blockbuster article on Harness Engineering. It's probably going to be the defining piece of this emerging discipline.

**Ryan Lopopolo** (1:19)
Thank you. It is fun to feel like we've defined the discourse in some sense.

**swyx** (1:24)
Let's contextualize a little bit this first podcast you've ever done. Yes. And thank you for spending it with us. Where is this coming from? What team are you in? All that jazz.

**Ryan Lopopolo** (1:32)
Sure, sure. I work on Frontier Product Exploration, new product development in the space of OpenAI Frontier, which is our enterprise platform for deploying agents safely, at scale, with good governance in any business. And the role of me and my team has been to figure out novel ways to deploy our models into package and products that we can sell as solutions to enterprises.

**swyx** (1:57)
And you have a background. I'll just squeeze it in there. Snowflake, Brex, Stripe, Citadel.

**Ryan Lopopolo** (2:02)
The exact kind of customer in my entire life.

**swyx** (2:04)
The exact kind of customer that you want to.

**Vibhu** (2:06)
So I'll say, I was actually, I didn't expect the background. When I look at your Twitter, I'm seeing the opposite, stuff like this. So you've got the mindset of like full send AI coding, stuff about slop, like buckling in your laptop on your Waymos. And then I look at your profile, I'm like, oh, you're just like, you're cracked in the other room too.

**Ryan Lopopolo** (2:25)
It's quite fun to be AI maximalist. If you're going to live that persona, OpenAI is the place to do it. And it's certainly helps that we have no rate limits internally, and I can go, like you said, full send at this thing.

**swyx** (2:38)
Yeah. So the OpenAI Frontier, and you're a special team within OpenAI Frontier.

**Ryan Lopopolo** (2:42)
We had been given some space to cook, which has been super, super exciting. And this is why I started with kind of a out there constraint to not write any of the code myself. I was figuring if we're trying to make agents that can be deployed into end enterprises, they should be able to do all the things that I do. And having worked with these coding models, these coding harnesses over six, seven, eight months, I do feel like the models are there enough, the harnesses are there enough where they're isomorphic to me in capability and the ability to do the job.
So, starting with this constraint of, I can't write the code, meant that the only way I could do my job was to get the agent to do my job.

**Vibhu** (3:25)
And like just a bit of background before that, this is basically the article. So, what you guys did is five months of working on an internal tool, zero lines of code, over a million lines of code in the total code base. You say it was Cenex more like, it was Cenex faster than you would have if you had done it by hand. So, that was the mindset going into this, right?

**Ryan Lopopolo** (3:46)
Started with some of the very first versions of Codex CLI with the Codex Mini model, which was obviously much less capable than the ones we have today, which was also a very good constraint, right? It's quite a visceral feeling to ask the model to build you a product feature and just not being able to assemble the pieces together, which kind of defined one of the mindsets we had for going into this, which is whenever the model just cannot, you always pop open the task, double-click into it, and build smaller building blocks that then you can reassemble into the broader objective. And it was quite painful to do this, honestly. The first month and a half was ten times slower than I would be. But because we paid that cost, we ended up getting to something much more productive than any one engineer could be, because we built the tools, the assembly station, for the agent to do the whole thing. But yeah, so onward to GPT-5, 5.1, 5.2, 5.3, 5.4. To go through all these model generations and see their quirks and different working styles also meant we had to adapt the code base to change things up when the model was revved. One interesting thing here is 5.2, the Codex Harness at the time did not have background shells in it, which means we were able to rely on blocking scripts to perform long horizon work. But with 5.3 and background shells, it became less patient, less willing to block. So we had to retool the entire build system to complete in under a minute. And this is not a thing I would expect to be able to do in a code base where people have opinions. But because the only goal was to make the agent productive, over the course of a week, we went from a bespoke make file build to Bazel to Turbo to NX and just left it there because builds were fast at that point.