ARCHIVE: GPT-3 Hype

**Frank Chen** (0:00)
What GPT-3 shows is that they train this model once, and then they throw at a whole bunch of natural language processing tasks, like fill in the blank, or inference, or translation, and without retraining it at all, they're getting really good results compared to finely tuned models.

**Derrick Harris** (0:22)
Thank you for watching. Hi, I'm Derek Harris, and you're listening to the a16z AI Podcast. We're weeding into all things artificial intelligence with our in-house team of experts, as well as the founders, engineers and researchers working at the state of the art.
In this episode though, we're traveling back in time to the distant past, in AI years at least, of 2020 Because amid all the news over the past 18 months or so, it's easy to forget that Generative AI, and LLMs in particular, have been around for a while. OpenAI released its GPT-2 paper in late 2018, which excited the AI research community, and in 2020 made GPT-3, as well as other capabilities, publicly available for the first time via its API. This episode dates back to that point in time. It was originally published in July of 2020, when GPT-3 piqued the interest of the broader developer community, and people really started testing what was possible. And although it doesn't predict the precambrian explosion of multimodal models, regulatory and copyright debate, and entrepreneurial activity that would hit a couple of years later, and who could have really, it does set the table for some of the bigger and still unanswered questions about what tools like LLMs actually mean from a business perspective, and perhaps more importantly, what they ultimately mean for how we define intelligence. So set your way back machine to the seemingly long ago summer of 2020, and enjoy a16z's Sonal Chokshi and Frank Chen discussing the advent of commercially available LLMs.
As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Sonal Chokski** (2:20)
This week, we're covering all the recent and ongoing buzz around the topic of GPT-3, the Natural Language Processing Based Text Predictor from the San Francisco research and development company OpenAI. They actually released their paper on GPT-3 in late May, but only released their broader commercial API a couple of weeks ago, so we're seeing a lot of excitement and activity around that in particular, although it's all being called GPT-3.
So we're gonna do one of our explainer episodes. It's a 2x explainer episode going into what it really is, how it works, why it matters, and broader implications and questions while teasing apart what's hype, what's real, as is the premise of this show. But before I introduce our expert, let me just quickly summarize some of the highlights. So while GPT-3 is technically a text predictor, that actually reduces what's possible because of course, words and software are simply the encoding of human thought, to borrow a phrase from Chris Dixon, which means a lot more things are possible. So we're seeing, and note these are all cherry picked examples, believable forum posts, comments, press releases, poetry, screenplays, articles. Someone even wrote an entire article headlined OpenAI's GPT-3 may be the biggest thing since Bitcoin and then revealed midway that he didn't actually write the article, but that GPT-3 did. We're also seeing strategy documents like for business, CEOs and advice written entirely in GPT-3, and not just words, but we're seeing people design using words to write code for designing websites and other designs. Someone even built a Figma plugin. Again, all of it showing the transmutability of thoughts to words to code to design and so on. And then someone made a search engine that can return answers and URLs in response to quote, ask me anything, which is anyone who's been in the NLP space knows.
I was at park when we spun off PowerSet back in the day, and that's always been sort of a holy grail of question answering, which you know all about too, having worked in this world, Frank.
Now, let me introduce you, our expert in this episode. Frank Chen has written a lot about AI, including a primer on AI deep learning and machine learning, a pulse check on AI, what's working, what's not, a microsite with resources for how to get started practically and do something with your own product and your own company, and then reflecting on jobs and humanity and AI working together. You can find all of that on our website. Frank, to start things off, what's your favorite example of GPT-3 so far? Mine is founding principles for a religion written in GPT-3. I'd love to hear your favorite, and also your quick take on why the excitement to start us off before we dig in a bit deeper.