Automating Software Engineering: Genie Tops SWE-Bench, w/ Alistair Pullen, from Latent.Space podcast

**Nathan Labenz** (0:00)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. This episode is brought to you by WorkOS. If you're building a B2B SaaS application, at some point, your customers will start asking for enterprise features like SAML authentication, SCIM provisioning, role-based access control, and audit trails. That's where WorkOS comes in, with easy to use and flexible APIs that help you ship enterprise features on day one without slowing down your core product development. Today, some of the hottest startups in the world are already powered by WorkOS, including ones you probably know like Perplexity, Vercel, Jasper, and Webflow. WorkOS also provides a generous free tier of up to 1 million monthly active users for user management, making it the perfect authentication and authorization solution for growing companies. It comes standard with rich features like bot protection, MFA, roles and permissions, and more. If you're currently looking to build SSO for your first enterprise customer, you should consider using WorkOS. Integrate in minutes and start shipping enterprise plans today.
Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to share a special crossover episode from the Latent.Space podcast, co-hosted by Swyx and Alessio. For any that don't already know, Latent.Space is the number one podcast for practical insights for AI engineers. And while longtime listeners may remember that The Cognitive Revolution was voted number three in the same poll, and I can at times be a bit competitive about these things, in this case, I have to agree with the wisdom of the crowd. The Latent.Space podcast is a regular must-listen for me, and I am grateful to Swyx for allowing me to share this episode here. I wanted to cross-post Swyx and Alessio's conversation with Alistair Pullen of Cosine, and Makers of Genie, in particular because it serves as such an excellent complement to my recent episode on AI automation. In that episode, I covered the core best practices needed to automate routine business processes. Which by the way, are pretty much exactly the same as you'd use to evolve a software application from DIY to done-for-you by AI. And this conversation really demonstrates just how far that paradigm can be pushed. Alistair and his team are implementing almost exactly what I outlined. They are very thoughtfully breaking complex work into smaller AI manageable parts, curating gold standard examples, using synthetic data generation to fill in critical reasoning chains, and fine-tuning intensively to achieve the desired AI behavior. They just happen to be doing it with an unusually high level of ambition, a significant budget for data curation, creation, and fine-tuning, and one major additional step of weaving all of the individual optimized workflows together into an AI coding agent. The key takeaway here is clear. If Cosine can achieve a new state-of-the-art on the popular software engineering benchmark, Swybench, with this approach, there is no reason to doubt that you can apply these techniques in your own business if you buckle down and do the work. With that in mind, as I recently mentioned, I'm exploring working with a few companies as an AI automation advisor. I've recently started one project helping a company evolve their application from DIY to done-for-you by AI, and I have bandwidth for maybe one or two more projects right now. So if you're interested, please reach out and we'll see if it makes sense to work together. As always, if you're finding value in the show, we appreciate it when folks take a moment to share it with friends, and we always invite your feedback and suggestions either via our website cognitiverevolution.ai or by DMing me on your favorite social network. And I really do encourage you to subscribe to the Latent.Space podcast if you haven't already. Another recent episode with Michelle Pokras of OpenAI, which includes a deep dive on the Structured Outputs API and other underappreciated features of OpenAI's APIs, was particularly valuable for me. I've been using some of the techniques that they discussed in just the last two weeks since that episode aired, and I have no doubt that Latent.Space will once again provide some of the best and most comprehensive coverage of this week's OpenAI Dev Day, which will again help me raise my game. Now, I hope you enjoy this glimpse into the cutting edge of AI automation with Alistair Pullen of Cosine from the Latent.Space Podcast.

**Alessio** (4:34)
Hey, everyone. welcome to the Latent.Space podcast. This is Alessio, Partner and CTO in Residence at Decibel Partners, and I'm joined by my co-host, Swyx, founder of Small AI.