#211 - Is AI Distillation Illegal?

Jacob Robinson (0:00)

Is it illegal to train an AI model by distilling from it, by asking it questions, and using its outputs, its answers that you've generated to train your own model? This is one of the most important legal questions for AI.

SPEAKER_2 (0:14)

A trillion dollar sell-off.

SPEAKER_3 (0:16)

Front and center this hour, the deep sell-off, a major tech sell-off today. Deepseek sell-off is still really pressuring tech.

SPEAKER_2 (0:23)

Stemming from growing fears around a technique that could upend the AI leaderboard.

Daniel Barabander (0:28)

A technique called distillation.

SPEAKER_5 (0:30)

Distillation. Their work is being distilled.

Jacob Robinson (0:33)

Distillation is pretty simple. I mean, it's complicated, but it's easy to explain. It's basically training a model the way you or I use ChatGBT. You ask it a bunch of questions and save every answer. And then you take all those answers, those question and answer pairs, and you train a smaller model to respond how the larger model did. So rather than invest in data centers, buy all the data, use all that compute, you're basically just taking that existing model and learning from it.

Daniel Barabander (1:03)

What these attacks fundamentally are doing, these isolation attacks, is that these are primarily, as alleged, Chinese labs that are pretending to be legitimate users, using things like proxies and something like transfer stations, which basically they are a mixer where they aggregate a bunch of requests from distillers, and then they mix in like legitimate traffic, and they do all this type of stuff to make it seem like it's basically load balanced, to make it seem like it's all like legitimate traffic, that people just like trying to use JackTvT or whatever.

Jacob Robinson (1:31)

Of course, the Frontier Labs aren't happy about this. Anthropic published a report claiming that three Chinese labs ran industrial scale campaigns to copy Claude's capabilities by distilling its outputs. In that report, they called this version of distilling, this type of distilling, that these Chinese labs were doing illicit. But is distillation illegal? Welcome to the Law of Code Podcast, where we unpack the legal layer of emerging tech. I'm Jacob Robinson. And by the end of this episode, I promise you'll understand not only how distillation works in the context of AI, but also how the law might apply. We go deep on existing case law, specifically a relevant web scraping case.

Daniel Barabander (2:13)

Okay, why did I just talk about this? It's a scraping case. It's not a distillation case. As you already pointed out, distillation and scraping are deeply related. They're both about accessing data. The party really doesn't want you to access when we're talking about distillation of a third party, not when labs distill their own models.

And I think that that's exactly how it's going to break down for these distillation cases as well.

Jacob Robinson (2:36)

You just heard from Daniel Barabander, one of the three people I interviewed for this episode. Daniel is an investment partner and the chief legal officer at Variant, and he is joined by Sabina Beleuz, deputy general counsel at Variant, and Elijah Fox, an investor at Variant. The three co-authored an excellent article on whether or not it is illegal to train a new AI model on a frontier model's outputs.

Daniel Barabander (3:02)

Put all of this together, why it's really important is distillation has been this key bulwark against these frontier labs.

Basically, if you look at what it makes to train a frontier model, it's really three things. It's compute, data, and talent.

And Anthropic and OpenAI, other frontier labs in the US here, they have all three. And these are businesses that are based primarily on economies of scale and brand power. So if you just zoom forward a little bit and you just think, okay, they already have infinite money, they already have infinite talent, they already have infinite compute and data, it's not that hard to see how this can become a runaway train where they just control frontier intelligence and have monopoly on it. So I think there's a truly existential question of whether or not it's legal, because if it is illegal, that's going to suggest that this monopoly will play out, which I think will be net negative.

Jacob Robinson (4:02)

Now I spend a lot of time putting together these podcasts. I actually flew to New York to interview the three guests in this episode. I'm able to do that thanks to the presenting sponsor of this episode, as well as the other sponsors as well. The presenting sponsor here today is Altitude. Altitude is a payments platform that's available in over 150 countries that lets you run all of your payments across every entity on one platform, thanks to Stablecoins. Now you can transact still with local vendors in their local currency while using that same platform. I actually spoke with some of their clients, people who use Altitude and wanted to share the benefits.