Harvey’s Margins, Are Insects Worth More Than Humans?, Meta Tests Human Help for Muse | Diet TBPN

John Coogan (0:01)

Jordi, his hat game has evolved. He's now wearing the fedora with the blackstone cap on top, which I assume can only mean one thing. It means that your top priority is safety in...

Jordi Hays (0:15)

Financial markets.

John Coogan (0:16)

Financial markets and the financialization of...

Jordi Hays (0:18)

But also my safety as well.

John Coogan (0:21)

Oh, you care about both. I care about both. So you want to accelerate the AI build out safely. Yes, I thought it was, I thought it was you're gonna blend the two, and you're like, look, I want insane private credit deals, trillion dollars of capital to flow into the AI build out, but I wanted to do it safely.

Jordi Hays (0:39)

I want to prioritize safety on the whole frontier. Okay. Safety around the models.

John Coogan (0:43)

The capital markets frontier? And the financial markets. In the capital markets.

Jordi Hays (0:46)

Got it.

John Coogan (0:47)

Okay, okay. I'm seeing where you get there. Anyway, you want to do Harvey first?

Jordi Hays (0:51)

Let's...

John Coogan (0:52)

People are taking shots at Harvey. It started with Samuel Nolan. I won't stand for it. I won't stand for it. No, I like Harvey.

We've talked to some lawyers that use Harvey. Seems like a great product. We also had LaGuard on the show just last week. Had a good time with them. Seems like a very competitive space. But the headline that grabbed attention, they say, track changes. Harvey's gross margin fell from about 50% to negative 50% by June as agent token use spiked 20-fold on rented open AI and anthropic models Bloomberg reports. And so it seems really bad. Like you definitely don't want negative gross margins as a software company, as a tech company. As any company. Really as any company actually, you don't want to be selling a dollar for 50 cents. And that's exactly what they did in June, basically. That's what they were doing. They were selling tokens at negative gross margins. Not good, but there's way more details in the actual reporting. So you should go read the Bloomberg report. The first thing is cost did explode, but it was on the back of more product usage, which is good. They're growing ARR. We'll get into that.

And token consumption increased 20x this year. So basically, they are feeling the reasoning era, the agentic era. One job, one process is generating a lot more tokens.

Jordi Hays (2:04)

Had seat-based pricing, and then AI got a lot better. So every seat was just using it way more. Very simple.

John Coogan (2:10)

But it's not just using it more. It's that the lawyer might be saying the exact same thing as they did a year ago. They might say, review this. But instead of one-shotting it with no reasoning, they went to reasoning, which generated way more tokens, and then they went to agents. So they're like, okay, it's actually gonna fan out, look at everything across your firm, all the other records that you have, all your different policies and MD files. Everything that you have is gonna be worked through this agent, and so you're generating way more tokens. So that created negative gross margins. But the good news is that they're already back to positive gross margins, which is in the Bloomberg article. Kind of criminal not to repurpose that, because it really makes it look like they are running at negative gross margins. They had a negative 50% gross margin in June, but they are back to positive gross margins. They changed their model usage, of course. There's a whole bunch of models that are cheaper and better for certain things. And they actually fine-tuned an open weight model. So they did post-training on an open weight model, and they're calling it Harvey Tenet. And there's some interesting details there. I want to debate with you on the value of that, where that goes.

Jordi Hays (3:16)

First, Gabe, one of the founders of Harvey, responded.

John Coogan (3:19)

Oh, he did.

Jordi Hays (3:20)

He said, the hardest thing about building, Harvey is doing what's best for our customers, despite immense pressure to do what's easy. The easy thing would have been to force our customers into consumption pricing before they were ready and serve them worse models to protect our margins. We chose to help our customers transition on a timeline that works for them and give them the best models in the meantime, even though it hurt our margins. This meant optimizing our product through routing, harness improvements and post-training so we could serve frontier intelligence at an affordable price. It also meant building the infrastructure for customers to monitor and manage spend, usage dashboards per matter, cost attribution, spend caps, and ROI reporting. As a result, we improved our gross margins from negative 50 percent to positive in a single quarter, despite usage doubling month over month and continuing to serve the best models. Our philosophy is simple, do what's best for our customers, even when it's painful, hurts our margin, or draws criticism from competitors X and the press. We believe the most important part of building a company is earning and keeping your customers' trust. You do that by doing the hard thing for them, even when it costs you.