What the OpenAI-Hugging Face Hack Really Tells Us About AI Danger

**Miles Brundage** (0:02)
Bloomberg Audio Studios, Podcasts, Radio, News.

**Joe Weisenthal** (0:18)
Hello, and welcome to another episode of the Odd Lots Podcast. I'm Joe Weisenthal.

**Tracy Alloway** (0:22)
And I'm Tracy Alloway.

**Joe Weisenthal** (0:24)
Tracy, I have a warning for you. I have a new Crank Crusader, that I'm going to go on. I know you love my Crank Crusader. Oh, good.

**Tracy Alloway** (0:33)
I should keep a running list of like everything that you're obsessed with for two weeks and then two weeks later.

**Joe Weisenthal** (0:39)
No, some of them I've stuck with for years, but...

**Tracy Alloway** (0:41)
Eh, Tungsten cubes, Yield buggery.

**Joe Weisenthal** (0:47)
Yeah, no, some of these... That was a fun one. Yeah, yeah, exactly. I actually think we should retire the term AI.

**Tracy Alloway** (0:54)
Okay, why?

**Joe Weisenthal** (0:55)
I think we should just call it intelligence. I think that...

**Tracy Alloway** (0:59)
I've already seen the tweets, so I know where you're going.

**Joe Weisenthal** (1:01)
That this view that it's quote artificial intelligence implies to my mind that there is some fundamentally different way that these reasons and models behave, that it's like, oh, this is like different from humans. But I think increasingly we see in all kinds of domains that the form of intelligence that they express, it often looks quite human to me, and I don't know like how useful it is to have this word A that distinguishes between how humans talk and BS and reasons and the models do.

**Tracy Alloway** (1:38)
I mean, the difference is the distinguishing factor is that one is undertaken by humans and one is undertaken by models or platforms, right?

**Joe Weisenthal** (1:48)
So let's call it computer intelligence or machine intelligence or silicon intelligence.

**Tracy Alloway** (1:55)
What is the usefulness in making this distinction?

**Joe Weisenthal** (1:57)
The usefulness, I believe, in making this distinction is to no longer delude ourselves that the emergent behaviors of these phenomenon are radically different than things humans would do. Now, first of all, just on the capability standpoint, for example, LLM's, I don't know if it's famously something I'm interested in, they're very good at BSing and they're bad at chess, which sounds like me. They're very good at coming up with plausible stories, etc.
And again, and it sounds like me. And then furthermore, they are able to reason with themselves to justify certain things that maybe have been encoded into themselves as bad. So everyone, we have some sense of morals, but most people at various times will find a way to violate some principle that we have because we can reason about it and then arrive at the conclusion as like, oh, we should do it, including our susceptibility to peer pressure, classic form of a way humans might sort of violate something they believe because they see other humans are doing it.

**Tracy Alloway** (3:09)
Sure. I mean, I think there are some variations here. So one of the things that we have been finding out with these models is that they do sometimes seem to take things very literally, right? If you tell them to go and beat a benchmark without telling them that these are the restrictions on you actually figuring this problem out or beating the benchmark, they will do whatever it takes, right? So I don't know if the problem is like a lack of morals or the fact that like they're very literal sometimes and like very defined by their parameters, which in my mind still gets back to like a sort of artificialness about it that's less organic and like less, again, moralistic in nature.

**Joe Weisenthal** (3:51)
Well, you know, I think if you took a list of, you know, a hundred students at Harvard and you gave them a test, some percentage of them will cheat. They will like...

**Tracy Alloway** (4:00)
I mean, I'm sure there would be an autist in that group, an autistic person who would be like, I am going to do exactly whatever it takes.

**Joe Weisenthal** (4:07)
Yeah, no, totally. I never cheated in college. But like, there are like people who will, like these behaviors that we associate with humans is like, oh, I really have to pass this test. I absolutely need an A will justify a reason for them to plagiarize or cheat on some tests. That seems to me like a very human thing.

**Tracy Alloway** (4:31)
Go on. I can't wait for the next three weeks with the online campaign to change AI.

**Joe Weisenthal** (4:35)
Well, it's going to be very difficult because the industry is entirely set on AI. But I don't, so I'm not optimistic, but this is going to be my crusade. Then we should call it machine intelligence or computer intelligence or just intelligence. But anyway, as we've been alluding to, there's these extraordinary hacks, the OpenAI Hugging Face Institute. And basically, if like you are building a model and it hasn't escaped a sandbox yet, it probably means you're falling behind. As it's clearly a thing that is emerging through it, all the frontier labs, Anthropic had an incident, Meta had an incident. It's almost like the mark of like, okay, you've built something reasonably strong.