The Data Factory: Inside the $100B Race for Post-Training Supremacy, with Labelbox CEO Manu Sharma

**Erik Torenberg** (0:00)
Hello, and welcome back to The Cognitive Revolution. Today, my guest is Manu Sharma, founder and CEO of Labelbox, a data factory that supplies frontier training data to all of the top Western AI labs and many enterprises that are pushing the performance frontier with task-specific, fine-tuned models. This conversation really couldn't be more timely. In the wake of Meta's recent $15 billion deal with Scale AI, which of course saw Scale's former CEO Alex Wang join Meta to lead their superintelligence team, other frontier model developers have been left scrambling to secure their own training data pipelines, and the market overall is still in the process of realignment. These headline-making developments have really spotlighted just how critical super high-quality training data is to today's frontier capabilities, and also how difficult and expensive it can be to create. Post-training budgets are growing rapidly as labs race to imbue their models with differentiated capabilities. And as Manu explains, every Western frontier lab is now spending over a billion dollars annually on frontier training data. Such tremendous investment is required because modern data work has moved far beyond the simple labeling and preference indication tasks that many are familiar with from years past. And so today, with Manu as our guide, we'll be tracing the evolution of post-training — from supervised learning from human examples, to reinforcement learning from human feedback, to the reinforcement learning from verifiable reward paradigm that's ascendant today, and unpacking how all of that has shifted data creation work away from tools that facilitated the collection of human preferences and reasoning, and toward environments, which Manu calls gyms, where models can develop new skills, starting with coding, mathematical reasoning, and computer use through a process of trial, error, and highly automated feedback. The bottom line is that today, when you think about training data creation, you should envision not the data sweatshop of the past, but highly qualified and comfortable domain experts working to create sophisticated reinforcement learning environments and verifiers that teach frontier models to solve complex, long-horizon tasks. A quick disclaimer before we begin. Labelbox will be sponsoring the show for at least the next month, and so this does qualify as a sponsored episode. Nevertheless, I can sincerely say that this is a conversation I'd have wanted to have anyway, because whether you're tracking AI progress analytically or trying to achieve superhuman performance in your particular AI application, understanding the best practices for training data creation is essential. And besides, the scale and scope of Labelbox's operation is genuinely remarkable. Their most recent capital raise was $110 million in early 2022, and today Labelbox operates as a vertically integrated data factory. They're scaling their expert network in part by conducting over 2,000 AI-powered interviews per day.
And their most in-demand experts are already earning more than $250,000 per year on the platform. I haven't yet had the chance to sign up to earn some of that money myself, but I am looking forward to the AI interview experience. And assuming Manu's right that the system does recognize me as a qualified expert, I'll report back, subject of course to any required NDAs, on my experience as an AI trainer. Now, without further ado, I hope you enjoy this deep dive into the ongoing evolution of the red-hot training data market, with Manu Sharma, founder and CEO of Labelbox. Manu Sharma, founder and CEO of Labelbox, welcome to The Cognitive Revolution.

**Manu Sharma** (3:29)
Thank you.

**Erik Torenberg** (3:30)
So, I'm excited for this conversation. Your world is in chaos. The AI world is going through a bit of a reorganization or realignment right now. And when I say your world, I mean the world of data, data creation, human sources of data.
You've been in this business with Labelbox for a number of years, and we'll have a chance to dig into all the different facets of that. But obviously, the news that has sent various forms of shockwaves, I think, through the industry in the last couple of weeks, is Zuck and Meta doing this weird deal with Scale, where they're getting Alex to come over and lead the super intelligence team or something along those lines. Then, of course, we've got people leaving Scale, and it seems like, generally speaking, it's chaos. So, what's your report from the front line? Is this driving a lot of opportunity for you? Are people confused? What are the takes that you are hearing that you think are interesting? Just super interested in your gonzo report from the front to get started.

**Manu Sharma** (4:26)
Yeah, so this is one of the most exciting times in the AI industry, just generally across the board, right? You can see when you kind of really look into the innovations and case of progress, I believe we are experiencing the maximum innovations we've ever seen in per day or per week kind of time period. And it's an AGI race. It's a race among a number of companies and groups. And it's really interesting to see there are like this big AI labs that are playing and pushing out awesome capabilities of base models as well as products and product experiences. But then you have a number of teams that are just emerging with some new ideas and taking a bet on alternative techniques and so forth.