OpenAI Invests $5M Into the Self-Driving Race with John Hayes, Cofounder of Ghost Autonomy

**Erik Torenberg** (0:00)
Turpentine is a network of podcasts, newsletters, and more covering tech, business, and culture, all from the perspective of industry insiders and experts. We're the network behind the show you're listening to right now.
At Turpentine, we're building the first media outlet for tech people by tech people. We have a slate of hit shows across a range of topics and industries, from AI with Cognitive Revolution to Econ 102 with Noah Smith. Our other shows drive the conversation in tech with the most interesting thinkers, founders, and investors, like Moment of Zen and my show Upstream. We're looking for industry leading hosts and shows along with sponsors. If you think that might be you or your company, email me at erik.turpentine.co, that's E-R-I-K at turpentine.co.

**John Hayes** (0:46)
What we're putting together is the car should tell you why it's doing what it's doing in plain language. If you've rented a car that has a bunch of features on it, it will beep at you and show you some symbol you've never seen and you don't know why it's beeping at you.
It has this very, very narrow communication path. I think the next evolution is like, yeah, you should just chat with your car and tell it what you want. Let's turn on absolutely everything. You tend to get sensible answers, and you can ask it to explain the answers and explains the answers in terms of visual elements of the scene. You're going to make a system that out of the box is dramatically more reliable. But hallucination is actually the best part of these models, because the hallucination is where all the common sense applies.
And so when you're asking to incorporate not just all the knowledge of the scene, but all sorts of the entire depth of knowledge that could be applied, you tend to get more answers that are closer to reality than what an engineer having to express their thoughts in a programming language would be able to express. I think the amount of creativity that's going to unlock, we don't even know the limits.

**Nathan Labenz** (1:54)
Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Eric Thornburg. Hello and welcome back to The Cognitive Revolution. Today, my guest is John Hayes, founder and CEO of Ghost Autonomy.
If you thought that the OpenAI news was exhausted after Monday's Dev Day, today's episode shows that you were wrong. Just two days later, OpenAI and Ghost Autonomy are announcing a new partnership, including investment from OpenAI's Startup Fund, to bring large multimodal models to the grand challenge of self-driving cars. Since its founding six years ago, Ghost Autonomy has raised more than $200 million to pursue a standalone software primary strategy, building an integrated driving system on commodity hardware, and working toward long-term strategic partnerships with car makers.
Now, today, as we all know, self-driving cars are on the road, but are not quite ready for primetime at scale, for a mix of reasons, both technological and sociological. On the technology side, one of the major challenges has been figuring out how to handle all the unusual edge cases that humans navigate with common sense. And that's where Ghost's new partnership with OpenAI comes in. On top of the many low-level, high-frequency, but fundamentally very narrow components that they've already built, the plan now is to use a large multimodal model, complete with general world knowledge and common sense reasoning ability, to make the high-level decisions and tell those lower-level systems what to do.
In this conversation, John shares an overview of the autonomy space, including the different kinds of technology stacks that companies have developed and the strengths and weaknesses of each, the details of their new plan to combine super specialists and highly generalist systems into a single human-level AI driver, and his perspective on some of the non-technical challenges that companies in this space still have to overcome. As noted in my recent conversation with Flo from Lindy, I am very much an accelerationist when it comes to self-driving cars, and I would love not only to see this technology work, but to see society embrace it for the life and time-saving value that it promises for all of us. As always, if you're finding value in the show, we'd appreciate it if you'd share it with a friend. And of course, we welcome your feedback. Our email is tcr at turpentine.co, and I am easy to find on just about every social network. For now, I hope you enjoy this window into the historical challenges and the new opportunities in self-driving with John Hayes of Ghost Autonomy. John Hayes, founder and CEO of Ghost Autonomy, welcome to The Cognitive Revolution.