[AIE Summit Preview #1] Swyx on Software 3.0 and the Rise of the AI Engineer

**SPEAKER_1** (0:06)
Ladies and gentlemen, it's the Latent Space Weekend Edition.

**Swyx** (0:09)
Woohoo!

**SPEAKER_1** (0:10)
This weekend is a special one, as we are gathering many of our former guests and over 10,000 of you for our very first AI Engineer Summit, both in San Francisco and on YouTube. We're all very excited about the summit, and we were even interviewed on a few of our fellow AI podcasts about it. We figured we would cross post those episodes over the weekend to help you prepare for the exciting lineup of speakers, even if you can't join us in person. First, we'll have an introductory episode recorded with Tejas Kumar of Pod Rocket, where we introduce the concept of an AI engineer to a generalist engineer audience and go over Swyx's recent conference keynote on Software 3.0.
While you are listening, there are two things you can do to be part of the AI engineer experience.
One, join the AI Engineer Summit Slack. Two, take the State of AI Engineering Survey and help us get to 1,000 respondents. Both are linked in the show notes, and we would really love to have you. Now here's Swyx's conversation about software, 3 and the AI Engineer Summit.

**Tejas Kumar** (1:14)
I'm really excited to get into your talk, especially because we've already talked a little bit about AI and adjacent things. But before we get into it, let's zero in on scope a little bit and talk about your talk, Software 3 and the emerging AI developer landscape. I've seen the term Web 1.0, Web 2.0, Web 3.0, Web 3.0, also like an outside of Web Dev, there's the industry 3 is a trending term. I'm curious, Software 3.0, where does that come from?

**Swyx** (1:39)
It doesn't have any lineage whatsoever with Web 3.0. So that's a very unfortunate comparison there, I think. But the origin actually comes from a very influential article created by Andrej Karpathy about six years ago called Software 2.0. He was basically trying to articulate the difference between hand-coded software, where we write every single line of code ourselves with if statements, loops, and whatever traditional coding paradigms, and then machine-learned code, where you write the layers of what the machine learning model should do, like the architecture of the model, and then you just run it through a lot of data in order to achieve weights. And the weights themselves are the encoding of the knowledge. So he was trying to articulate that difference, that the possible space of problems you can tackle with Software 1 is the problems that you can code for deterministically, and the possible space of problems that you can address with Software 2 is the stuff that you can address it by machine learning. For example, computer vision and voice recognition. It's that stuff that you'll never be able to hand code by yourself. And I think there's the fundamental realization that a lot of people should have with regards to how they write Software 1 code, which is a lot of the times, what do you do as a programmer, as a software engineer, right? You write some functioning app and then you send it out there. You look at your analytics and your metrics and all that, and then you adjust by adding in some features and adding in some if statements and all that from learning. And essentially what Software 2 is accelerated learning from data, whereas in Software 1.0, we learn from data through humans in the loop and designers in the loop.
So I think that's a really fundamental realization there, that once you realize that sometimes you are just a very slow machine learning model, and you're writing all these algorithms, but yourself, sometimes you can just kind of machine learn the algorithms rather than writing them yourself.
So how do you perceive for Software 2 to 3 is the arrival of foundation models, and that's the change that has happened more or less in the last three years, enabled by the transformer architecture becoming a thing, which enables deep learning that's parallelizable and master scale, and obviously a lot more money and GPUs and data thrown at this problem.
So now foundation models mean that you do not have to collect a whole bunch of data to create models before you start delivering ML products into production. You can just grab one off the shelf, whether it's open source or closed source, it doesn't really matter. You take a foundation model and then you put that into production, and then you can start collecting data to fine-tune them if you want to, but otherwise the time to MVP of an AI product has significantly by reduced by orders of magnitude in the Software 3 paradigm.