Exploring AI's Pace of Evolution, AGI's Future, and Data Dominance with Adept CEO David Luan | E1855

**Jason Calacanis** (0:00)
How far away are we from AGI in your mind?
I turn on my computer in the morning, I go to work, you know, whatever I happen to name my assistant, Joe, and I say, hey Joe, what should I work on today? And Joe says, well, you know, looking at your email box, there's seven companies that have acute issues in your portfolio, and these three probably require a phone call. These four, you probably need some more information based on what I've learned, so I'm gonna send them requests for this information and then schedule them for tomorrow and Wednesday. Is that okay? Boom, and it just like kind of tells me what I'm doing for the next three days. When will that happen?

**David Luan** (0:33)
I was gonna ask you how you were gonna define AGI, and if you define AGI by just what you said in that flow, I think that we'll be at a spot where you would be able to get that within the next one to two years, but would you trust the recommendations?

**SPEAKER_3** (0:47)
This Week in Startups is brought to you by Brave. If you're building AI and search-based applications, train your models with the Brave Search API. Get started for free at brave.com/jason. Arising Ventures is a holding company that acquires tech startups facing setbacks. Arising Ventures knows what founders care about because they aren't bankers, they're tech founders themselves.
Go to arisingventures.com/twisttoday to learn more and connect with the team.
And LinkedIn jobs. A business is only as strong as its people and every hire matters. Post your first job for free at linkedin.com/twist.

**Jason Calacanis** (1:29)
Hey everybody, welcome back to This Week in Startups. We're continuing our in-depth coverage of AI. It's moving at a crazy pace and you're in for a treat today because we have David Luan, who is the CEO and co-founder of Adept AI. Previously, David ran research and engineering at a little organization known as OpenAI back in 2017 He left there to go work on large models at Google where he focused on Google Brain.
And if you're wondering what Adept AI is, we're gonna learn all about that today. But basically they're building a machine learning model that can interact with everything on your computer. Welcome to the program, David.

**David Luan** (2:04)
Thanks so much for having me.

**Jason Calacanis** (2:06)
Okay, so you are not the highest profile person at OpenAI, but you were a very key person. Maybe you could explain by background what you worked on at OpenAI because it's pretty darn impressive.

**David Luan** (2:18)
Thanks.
So my time at OpenAI was really engaging and fun. I knew a bunch of the core researchers there from just the very tiny machine learning research community from back in the day. Like the analogy I always like to make about how ML used to work is, imagine a world where the flat earthers toiled in obscurity for decades and then turned out to be right. That's basically the story of deep learning. And so that community of deep learners are actually pretty small. And I joined with OpenAI when I was about 35 people and ultimately grew it to about 135 before I left.
And primarily folks in my org covered basic research. So things like Qt2 and Clip and Dali all the way to the supercomputers and some of the larger scale up efforts there as well.

**Jason Calacanis** (3:03)
So when you were building those language models, and maybe you could talk a little bit about what they were trained on. I know there were collections of data sources, like there's the open crawl of the web, there are image libraries that were put together. How was that original data set organized when you were in that like two and three phase?

**David Luan** (3:24)
So the thing about GPT-2 that I think most people don't recognize as being two of its core contributions. The first one is actually not data set related, but just real quick, it's this idea that every single natural language understanding task could be reframed as simply writing more text. So historically, people were training these models for like sentiment analysis of tweets and all that stuff, and you're training a model with the input as the tweet and the output as a score of as a positive or negative, then you just end up with this constellation of models that all do different things. But GPT-2 said we can just boil this all down to one objective, which is like write more text and the next word is, is it a positive or negative tweet? And you get it right. And the reason why that works is actually because of the data set. So historically, people training language models use things like Common Crawl, as you mentioned, which is like effectively, like it was initially made for making open search engines, right? It's just all the websites you can find on the internet, but most of them are trash. Like we looked at my colleague, Alec Radford, who was the lead author and I like looked at all the data and there would just be webpages and webpages of thousands of product codes for Sony cameras and stuff like that.