AI-Powered Filmmaking with Waymark's Stephen Parker and Josh Rubin

**Nathan Labenz** (0:00)
Turpentine is a network of podcasts, newsletters, and more covering tech, business, and culture, all from the perspective of industry insiders and experts.
We're the network behind the show you're listening to right now.
At Turpentine, we're building the first media outlet for tech people by tech people. We have a slate of hit shows across a range of topics and industries, from AI with Cognitive Revolution, to Econ 102 with Noah Smith. Our other shows drive the conversation in tech with the most interesting thinkers, founders, and investors like Moment of Zen and my show Upstream. We're looking for industry leading hosts and shows along with sponsors. If you think that might be you or your company, email me at erikaturpentine.co. That's E-R-I-K at turpentine.co.

**Josh Rubin** (0:46)
This stuff, it was exceptionally hard, maybe even harder than a traditional animation. You're working with an unknown artist in the room, who can give you exactly what you want or who can give you some random wonderfulness or some, as other people say, grotesque image.
So it's a huge challenge. In order for it to be good, there needs to be like a big 500-foot human vision.

**Nathan Labenz** (1:16)
Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. Today, I'm thrilled to be speaking with my longtime friends and teammates, Stephen Parker and Josh Rubin, creative leads at Waymark and creators of The Frost, a groundbreaking short film made entirely with Dall-E 2-generated imagery. For a bit of context on the AI journey that Stephen, Josh, and I have been on together, from 2017 to 2021, Waymark had built the easiest-to-use video creation app on the market, and the quality of the video templates that Stephen, Josh, and the creative team produced was our standout feature. However, feedback showed that users wanted more than an easy-to-use DIY solution. What they really wanted was an app that could create content for them. Now, I had been interested in AI forever and was always looking for AI tools to enhance our product, but I'd only done a tiny bit of hands-on development because, frankly, none of the AI technology available at the time really worked for our purposes.
That first started to change with OpenAI's release of GPT-3, and in September 2021, when we successfully fine-tuned the Curie model for the first time, I became convinced that generative AI was the solution. Some on the team, I think, thought that I'd lost it when I used my prerogative as CEO to pause just about everything else we were doing, up to and including board meetings, to organize a generative AI 101 crash course for the team and reorient our product roadmap entirely around generative AI.
Stephen and Josh, to their credit, came along for the ride and have since caught the AI wave in their own unique way. While I've got my 10,000 hours of AI usage with a mix of language, computer vision, and text-to-speech models, Stephen and Josh have gone super deep on AI art. We got first wave access to Dall-E 2 as an OpenAI Innovation Partner customer in early 2022, and since then, Stephen has personally generated over 1 million images with Dall-E 2 The result? The Frost is not just a proof of concept, but a legitimate 12-minute film with a coherent narrative and a consistent aesthetic.
In this episode, we get a behind-the-scenes look at their creative process, a sense for the challenges they face and the strategies they use to overcome them, and overall, one of the most sophisticated accounts of the current state of AI art that you'll hear anywhere from a team using these tools with the highest level of taste, vision, and skill. Creating The Frost was not easy, but this project does show how transformative generative AI is likely to be as it continues to mature. Only seven people are credited on this film. Josh and Stephen are Waymark team members Tommy Herman, Zach Poley, and Lexi Dietz, and collaborators Matt Session and Robert McFalls.
As always, we appreciate your reviews and your online shares, but this time I really want to encourage you to watch the film itself, and also the trailer they've recently released for The Frost 2, which they are already making with an entirely new generation of AI tools. We'll have links to these in the show notes and also to some visual behind the scenes content and an MIT tech review write up of the film as well. Now, please enjoy this fascinating conversation with Stephen Parker and Josh Rubin, AI creative pioneers and makers of The Frost. Josh Rubin, Stephen Parker, welcome to The Cognitive Revolution.