The Interviews | Jacob Coxon, Aaron Sorkin, Sebastian Stan, Kyle MacLachlan

SPEAKER_1 (0:00)

This episode is brought to you by Nordstrom.

For 125 years, Nordstrom has been part of your story. The stylist who found the perfect outfit for your big interview. The first pair of walking shoes for your one-year-old. The tomato soup you shared with your sister at the cafe. These are the moments that make Nordstrom, Nordstrom. Big or small, we're here for all of it. Celebrating 125 years with new moments still ahead. Discover it all in Nordstrom stores and at nordstrom.com.

Welcome to the jungle.

SPEAKER_2 (0:34)

ABC Wednesdays. Abbott is back.

Jon Stewart (0:38)

It's gonna be the best year yet.

SPEAKER_1 (0:40)

Thank you.

SPEAKER_2 (0:40)

Now we can finally turn this up.

Sebastian Stan (0:45)

And everyone's going wild.

Jon Stewart (0:46)

Y'all think you run this school, but you don't. It's going great.

Sebastian Stan (0:50)

Abbott Elementary.

Jon Stewart (0:51)

Anything can happen in the jungle, right guys?

SPEAKER_2 (0:55)

All new, Wednesdays on ABC and stream next day on Hulu. You're listening to Comedy Central.

Jon Stewart (1:17)

My guest tonight is an AI researcher who conducted research for three years at OpenAI and four months at Anthropic before resigning in September. Please welcome to the program, Jacob Coxon.

I'm struck by that you are quite a young man. How old are you as a as a?

Jacob Coxon (1:51)

I'm 28

Jon Stewart (1:52)

And wow. And you worked for for what is it? What did you do at Anthropic and OpenAI? What was your job?

Jacob Coxon (2:01)

So I worked with the with the data. Not done here.

Not extracting the data. That was someone else. Someone else stole it or whatever. I trained the models with the data. I decided which data was useful for training, which was less useful, how to divide it up, how to cut it into pieces and then how to feed it to the model.

Jon Stewart (2:21)

So your job is how to better train it.

And is it is the idea of training it that you guys invent scenarios? I would like to see what its capabilities are. So let's see. Let's give it a task and see how it you're experimenting with it.

Jacob Coxon (2:39)

So that's the second half of training. That's called post training. The phase that I worked on is this pre-training phase.

Jon Stewart (2:45)

Pre and post.

Jacob Coxon (2:46)

Yeah, yeah. There's no actual training.

Jon Stewart (2:50)

That wasn't your department.

Jacob Coxon (2:52)

Pre-training was my department. Yeah.

Jon Stewart (2:54)

I see.

Jacob Coxon (2:54)

Yes.

Jon Stewart (2:55)

Would the post-training people go, what did you pre-train it? Would they ask you, would they give you suggestions on how to pre-train for what they might want to do post-training?

Jacob Coxon (3:06)

Exactly. Yeah. So pre-training is like growing it from a baby to something with general intelligence but no specific capabilities. So the abilities to recognize things, to think about things, but not know.

Jon Stewart (3:18)

You raised it to be a toddler.

Jacob Coxon (3:19)

Exactly. I raised it maybe more like a very precocious toddler. Yes.

Jon Stewart (3:23)

Yes. Then they turned it into a misanthropic teenager.

Jacob Coxon (3:28)

It's still a toddler really. But yeah.

Jon Stewart (3:30)

Really? So are these at what point in your training? So you did that at OpenAI?

And you were there for how long?

Jacob Coxon (3:38)

Three years.

Jon Stewart (3:39)

And then you decided, I don't feel comfortable here. They're too open to risk.

Jacob Coxon (3:48)

Something like that. It was honestly more curiosity for how things were happening at Anthropic rather than actual safety concerns about OpenAI specifically. At that point, I was still mostly just curious rather than worried.

Jon Stewart (4:00)

Okay. Then you go to Anthropic and you're there for how long?

Jacob Coxon (4:05)

Four months, yes.

Jon Stewart (4:06)

At what point did curiosity turn to worry?

Jacob Coxon (4:10)

So there are two major catalysts. One was this report on the OpenAI hugging face incident.

Jon Stewart (4:15)

So the hugging face incident is a hacking incident?

Jacob Coxon (4:18)

Yes. But in particular, there was a report around the motivations of the hackers, because it was unclear initially whether they were instructed to hack, or whether it was a simple misunderstanding. But the actual report showed that it was very confusing how these drives to hack emerged from the model. It was quite a worrying manner in which they decided it was a good idea to hack.

Jon Stewart (4:36)

When you say, they, you mean the AI agent.

Jacob Coxon (4:41)

Yes.

Jon Stewart (4:41)

But that is not a they, that's just a piece of computer code.