The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic

**Alessio** (0:03)
Hey, everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel Partners, and today we're in the new studio with my usual co-host, Sean from Small AI.

**Swyx** (0:14)
Hey, and today we're very blessed to have Eric Schluntz from Anthropic with us. Welcome.

**Erik Schluntz** (0:19)
Hi, thanks very much. I'm Eric Schluntz. I'm a member of technical staff at Anthropic, working on tool use, computer use, and SWEBench.

**Swyx** (0:27)
Yeah. Well, how did you get into just the whole AI journey? I think you spent some time at SpaceX as well.

**Erik Schluntz** (0:35)
Yeah.

**Swyx** (0:36)
And robotics. Yeah. There's a lot of overlap between the robotics people and the AI people, and maybe there's some interlap or interest between language models for robots right now. Maybe it mentions a little bit of background on how you got to where you are.

**Erik Schluntz** (0:49)
Yeah, sure. I was at SpaceX a long time ago, but before joining Anthropic, I was the CTO and co-founder of Cobalt Robotics.
We built security and inspection robots. These are five foot tall robots that would patrol through an office building or a warehouse looking for anything out of the ordinary. Very friendly, no tasers or anything. We would just call a remote operator if we saw anything. We have about 100 of those out in the world, and had a team of about 100 We actually got acquired about six months ago, but I had left Cobalt about a year ago now because I was starting to get a lot more excited about AI. I had been writing a lot of my code with things like Co-Pilot, and I was like, wow, this is actually really cool. If you had told me 10 years ago that AI would be writing a lot of my code, I would say, hey, I think that's AGI. I realized that we had passed this level, like, wow, this is actually really useful for engineering work. That got me a lot more excited about AI and learning about large language models. I ended up taking a sabbatical and then doing a lot of reading and research myself and decided, hey, I want to go be at the core of this and joined Anthropic.

**Alessio** (1:53)
And why Anthropic? Did you consider other labs? Did you consider maybe some of the robotics companies?

**Erik Schluntz** (2:00)
So I think at the time, I was a little burnt out of robotics. And so also for the rest of this, any sort of negative things I say about robotics or hardware is coming from a place of burnout. I reserve my right to change my opinion in a few years. Yeah, I looked around, but ultimately I knew a lot of people that I really trusted and I thought were incredibly smart at Anthropic. And I think that was the big deciding factor to come there. Like, hey, this team's amazing.
They're not just brilliant, but sort of like the most nice and kind people that I know. And so I just felt like it could be a really good culture fit. And ultimately, like, I do care a lot about AI safety and making sure that, you know, I don't want to build something that's used for bad purposes. And I felt like the best chance of that was joining Anthropic.

**Alessio** (2:39)
In front of the outside, these labs kind of look like huge organizations that have this like obscure ways to organize. How did you get, you joined Anthropic, did you already know you were going to work on like SWEBench and some of the stuff you publish? Or you kind of join and then you figure out where you land? I think people are always here to learn more.

**Erik Schluntz** (2:57)
Yeah, I've been very happy that Anthropic is very bottoms up and sort of very sort of receptive to whatever your interests are. And so I joined sort of being very transparent of like, hey, I'm most excited about code generation and AI that can actually go out and sort of touch the world or sort of help people build things. And, you know, those weren't my initial projects. I also came in and said, hey, I want to do the most valuable possible thing for this company and help Anthropic succeed. And, you know, like, let me find the balance of those. So I was working on lots of things at the beginning, you know, function calling, tool use, and then sort of as it became more and more relevant, I was like, oh, hey, yeah, like, let's, it's time to go work on encoding agents and sort of started looking at SWEBench as sort of a really good benchmark for that.