The Quiet Exception Conundrum

SPEAKER_1 (0:01)

Hey, what's going on, everybody?

Brian (0:02)

Welcome to another Saturday Conendrum. I'm Brian, one of the co-hosts of The Daily AI Show. And every Saturday, I bring you a brand new conendrum, which is just really a debate and something for you to think about. It's not about solving the problem. It's not about me coming up with a solution or telling you what I think. It's just simply about providing two well-organized sides of a debate. And then ultimately, you come out wherever you come out on. A lot of times for me, it's I'm still on the fence. So this week, it's all about the ASI constitution conundrum. It comes off the idea that came off of a paper from last Saturday, actually, from Dario Amodei at Anthropical, We Must Pace the Frontier. Now we talked about that particular blog and everything that came after it and where people landed on it. Most of this week on the show, it came up quite a bit. But I want to take a little bit different approach to it and talk about not what we need to do to pace the frontier, but at some point in the future, I think we can all agree at some point in the future, there will be some version of an ASI. ASI is an artificial super intelligence. That would be one step above AGI, artificial general intelligence. It's a point at which you've got recursive self-learning from the AI. It is not just generally smarter than most humans. It is now the smartest thing we have on Earth.

At a point where ASI exists, how do you deal with the constitution of the world? There's really two different sides to this. There's the single moral framework argument for that.

They would argue that side, it's about global safety and minimum constraints. Then you have the multiple constitutions emphasizing more democratic legitimacy, right?

Meaning each country might have their own version of ASI, which means that a whole bunch of different types of ASIs with a whole bunch of different types of conundrum or constitutions have to work with each other, not unlike how diplomacy and countries work with each other today.

What's going to be the type of ASI that works best for the world? Assuming, and in this particular case for this conundrum, we're going to assume that ASI will be here one day. So let me give you the quick, or not so quick, but it could intro to it as well as the conundrum. And then as always, I'm going to hand it off to our two AI co-hosts. I always say this most weeks, if you want to know how I do this, well, I do the show five days a week most weeks. So I usually come up with some good ideas for a conundrum or one is sort of brought to me. And then I do research on it. I decide, yep, I think that's something I would like to learn or think about or here, basically, I think of myself as being the audience and I go, do I want to listen to a podcast about this or do I want to listen to two AI co-hosts debate full sides of this for 20 minutes? I mean, that's basically it. And if I go, yeah, I would actually, then that's kind of my indicator that this is a good idea. And then I do deep research. I've been doing that lately using Codex versus perplexity. But that's just because perplexity was doing some odd things a couple of weeks ago and sort of moved off of it. And then from there, it's fed into Google or Gemini Notebook, is what it's called now instead of Google Notebook LM. And then that gives me a well-defined, or I have the well-defined bait, I bring it into the notebook and then that's where we output our two AI co-hosts that follow me.

As you can tell, I'm not AI. I trip over enough words, I think, on the show in between doing this, so most people know that, but, you know, so this is me intro-ing it, and then we have our two AI co-hosts. So here's where, let's kind of get into this. So we talked about Dario's blog.

The CEO argued that AI capabilities are beginning to advance faster than our ability to understand and control them, repose independent evaluators, coordination among frontier labs in democratic countries, and eventually agreement with China. So there's a whole bunch that came out of that, I'm gonna kind of skip over that part about who was on what side, because really what matters is like underneath the argument sits a harder problem. Amodei repeatedly talks about improving alignment, the effort to make powerful AI systems behave according to human intentions and values. Anthropic even describes principles embedded in Claude's Constitution. But the more capable the intelligence becomes, the harder the obvious question is to avoid, whose values are we aligning to? Humans do not have one moral operating system, values different across nations, religions, political systems, generations, cultures, communities and geography. Historical experience changes what people mean by fairness, freedom, security, family justice, and even individual rights. Even with one country, people can disagree fiercely about what those principles should prevail and when they collide. Perhaps an ASI could be given a thin Constitution that sits above differences. Protect human life, don't destroy the planet. Do not deliberately cause human extinction, preserve human autonomy.