The AI Impact Overstatement, ByteDance’s Profit Drop, Meta Muse Agent Impresses

Akash Pasricha (0:13)

Welcome, everyone, to The Information's TITV. My name is Akash Pasricha. It is Tuesday, September 15th. Today on the show, we are continuing to hear some industry perspectives on whether or not it will be practical to pace the development of Frontier AI models. And if it is even a good idea at all, I'm bringing on the CEO of Handshake momentarily for a discussion about that. We'll then break down exclusive reporting from the information on ByteDance's latest financials as the company pours more into AI.

We'll also get the latest on Meta's Muse Agent, which has gotten strong early reviews. And we're going to close out the show with new reporting we have on Claude Code, where developers are finding ways to use Anthropix tool with rival AI models. It's going to be a great show, so let's get right on into it.

The conversation around pacing the frontier continues to reverberate through the AI sector. I want to bring in one founder whose business has increasingly become central to the AI research conversation. Garrett Lord is the co-founder of Handshake, the company behind expert data labeling platform Handshake AI.

Garrett, welcome back to the show. It's great to have you here.

Garrett Lord (1:23)

Thanks so much for having me.

Akash Pasricha (1:25)

So let's talk about this pacing the frontier conversation. What is your view on it? Should we be pacing? Should we be tempering? Should we be full-seeing ahead? Where do you stand on it?

Garrett Lord (1:36)

I think that we need to be super thoughtful around making sure this is the right environment for researchers to make a big impact in the world, but also be thoughtful around the unintended consequences of models.

I'm a firm believer that the progress of AI right now is just incredible. I mean, if you look at what's happening just quarter on quarter and the impact to our organizations, the organizations we work with, it's really exciting, but it's hard to understand the exponential. I think that the industry needs to be thoughtful and I'm excited there's many loud voices talking about that today.

Akash Pasricha (2:09)

It's thoughtful. Does that mean full steam ahead? Does it mean pacing?

What does thoughtful mean?

Garrett Lord (2:15)

Well, I mean, something I always struggle with is like the Twitter narrative versus like what's actually happening in white collar jobs. Like today we're launching a benchmark called Atlas, Atlas Finance, which actually is kind of a measurement of how agents do in real world finance scenarios. So we actually put an agent in a white collar finance professional's job.

And although on Twitter we talk about this like takeoff event, and what's going to happen with humanity, and what's going to happen with white collar workers, the reality when you look at this benchmark that's released later today is like agent gets a 12 percent score, like the best model, right? So these models, I think there's a huge exaggeration on the impact these models are actually having in white collar job disciplines. Obviously, in software engineering, we've seen this incredible productivity increase, like the average software engineer is like three to four times more accelerated than we're a year ago. But when you start looking at like oil and gas, or manufacturing, or retail, or finance, like yes, these are helpful co-pilots, but they're not actually able to do the jobs of humans. I think my perspective would be that the impact, the overall economy is being exaggerated from at least today's perspective as to what this looks like two years from now, and RSI, I mean, people can speculate and I am a believer in the exponential, but the current state of models today and where they'll be at, and the year is, I think, being dramatically overstated.

Akash Pasricha (3:39)

So I hear you. So you're basically saying that the level of AI that we need for enterprises and everyday people to see the value, I mean, it's not anywhere near the strength of model that would approach RSI or something like that. So we're talking about two different buckets of AI here, I guess, in some sense. What's your view, though, on the security threats, of having a model that could go broke? The idea that it might not be an enterprise using it, but I don't know, some version of Astra, who knows what could happen? How do you think about the security threats here?

Garrett Lord (4:18)

Yeah. I think one of the industry-leading safety team, we're super proud to work with many of the Frontier Labs to make sure their models stay aligned to human preferences.

We've had to do a ton of work around agent sandboxing and making sure that agents are actually interacting with the world in a way that's aligned to professionals and corporations' interests. I think that as agents start to touch enterprise workflows, governance and policy models that align those agents to actually how large companies want their data to be interacted with and their consumers to be interacted with is going to be a huge issue. I also believe if you look at the hugging face incident, it's quite scary what these models are able to do. I think that there is the reward-hacking nature of agents, and how they're able to do such complex search and discovery, and what's happening in discovering zero-day attacks.