Joseph Carlsmith - Utopia, AI, & Infinite Ethics

**Dwarkesh Patel** (0:06)
Today, I have the pleasure of interviewing Joe Carlsmith, who's a senior research analyst at Open Philanthropy and a doctoral student in philosophy at the University of Oxford.
Joe has a really interesting blog that I got to check out called Hands in Cities. And that's the reason that I wanted to have him on the podcast, because it has a bunch of thought-provoking and insightful posts on there about philosophy, morality, ethics, the future. And yeah, so I really wanted to talk to you, Joe, but do you want to give a bit of a longer intro on what you're up to? Sure.

**Joseph Carlsmith** (0:42)
So I work at Open Philanthropy on existential risk from artificial intelligence. And so, you know, I think about what's going to happen with AI, how can we make sure it goes well, and in particular, how can we make sure that advanced AI systems are safe?
And then I have a side project, which is this blog, where I write about philosophy and the future and things like that. And that emerges partly from sort of my background, which is I was before getting into AI and working at Open Philanthropy, I was in academic philosophy.

**Dwarkesh Patel** (1:19)
That's quite an ambitious side project. I mean, given the length and the regularity of those posts, it's actually quite stunning.
Do you want to talk more about what you're working on about AI at Open Philanthropy?

**Joseph Carlsmith** (1:33)
So it's a mix of things. Right now, I'm thinking about AI timelines and what's called takeoff speed, sort of how fast the transition is from pretty impressive AI systems to AI systems that are kind of radically transformative.
And I'm trying to use that to provide more perspective on the probability that everything goes terribly wrong.

**Dwarkesh Patel** (1:52)
I see.
I didn't know, but what are the implications? I suppose it's higher or lower than I would expect. I guess if it's higher, maybe I should work on AI 11 But other than that, what are the implications of that figure changing?

**Joseph Carlsmith** (2:06)
I think there are a number of implications just from understanding timelines with respect to how you prioritize and what just to some extent, the sooner something is, then you need to be planning for it coming sooner and cutting more corners or counting less on having more time. I think overall, the higher you think the probability of catastrophe is, the easier it is for this to become the most important priority. I do think there is a range of probabilities where it maybe doesn't matter that much. But I think the difference between say, one and 10 percent I think is quite substantive, and the difference between 10 and 90 is quite substantive.
I know people in all of those ranges.

**Dwarkesh Patel** (2:52)
Okay. Interesting.
Let's back up here and talk a bit more about the philosophy motivating this. I think you identify as a long-termist. Maybe a rough picture question here is, you have an interesting blog post about why the future looking back on us might think about the 21st century given the risk we're taking. What do you think about the possibility that we're potentially giving up resources, potentially dedicating, well, I'm not, you're dedicating your career to building a future that maybe, given the fact that you're alive now, you might find strange or disturbing or disgusting. I guess to add more context to the question, from a utilitarian perspective, the present is clearly much, much better than the past.
But somebody from the past might think that, there's a lot of bad things about the present that are kind of disturbing. I mean, they might not like the configuration of how maybe isolating a modern city might be, they might find that kinds of free to cheap information that you can access on your phone, kind of disturbing. Yeah. So how do you think about that?

**Joseph Carlsmith** (4:00)
So yeah, a few comments there. So one, I do think that if you took, you know, for most people throughout history, if you brought them to the present day, they would, my guess is that fairly quickly, and depending on exactly the circumstances, they would come to prefer living in the present day to the past, even if there are sort of a bit of future shock and a bit of some things are alienating or disturbing.
And, but that said, I think the distance, the sort of gap between historical humans and the present is actually much, much smaller, both in terms of time and kind of other factors than the gap I envision between present day humans and the future humans who are living, living ideally in a kind of radically better situation. And so I do expect sort of greater distance and possibly greater alienation. When you first show up, my personal view is that the best, the best futures are going to be such that if you really understood them and if you really experience what they're like, which may be a big step and might require sort of extensive engagement and possibly sort of changes to your capacities to understand and experience, then you would think it's really good.