AI:AM #4: Cameron on Model Consciousness, Duvenaud's Gradual Disempowerment, swyx's AI-Eng Alpha

**Nathan Labenz** (0:01)
Welcome to the AI in the AM Weekly Highlights, a cut for people who fall the frontier closely and can't watch every morning live. If you're new, AI in the AM is a live show that Prakash and I host most weekday mornings, at least through June, out of a studio Prakash Vibe Coded. The booking, the research, the editing, those are AI skills we refine as we go, and we publish them as they mature.
This week's conversations kept circling one question. How much do we actually understand about what's inside these systems and where they're taking us?
We start inside the model and zoom out from there. If something here is useful or something's off, tell us.

**David Duvenaud** (0:34)
That's how this gets better.

**Nathan Labenz** (0:36)
Let's go.
The Cognitive Revolution is brought to you by Mercury, the fintech that more than 300,000 ambitious companies and individuals trust to run their finances. I've wired AI into nearly every corner of my life. My email, my messages, my calendar. I even gave Mercury virtual cards to my agents, with low limits and category and merchant restrictions, for their autonomous use. But still, my AI's access to my financial data has remained limited. With a normal bank, I might export a bunch of statements and have my assistant process them for me. But for real-time, up-to-date information, and certainly for taking any action, trying to get your agent to use the bank via the browser is just too hard, too slow, and too error-prone to be worth it. And that's why Mercury's new conversational interface, Command, is such a big deal. It's built directly into Mercury, which means you get natural language access to your finances without exposing anything outside of your bank account. No exports, no spreadsheets, no pasting your transactions into third-party tools.
I really think a lot of people are going to prefer it this way. And it can already help you take actions too, with everything bound by the permissions and approval policies that you've already set up in your account.
I am genuinely impressed to see this level of AI integration in banking in 2026 And so I invite you to join me in the future. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and column NA, members FDIC. Thank you to Mercury for supporting the Cognitive Revolution. And now, on with the show.
We open with Cameron Berg, who studies artificial consciousness. He runs a lab called Reciprocal Research, where he designs experiments to test whether today's AI models have anything like inner experience, and how you'd even measure that. We started with the first order question, is consciousness all or nothing, or a matter of degree? And if it's a matter of degree, can you put a number on where a given model falls? Here's how he answers.

**Cameron Berg** (2:50)
The analogy that I reach for here is something like a dimmer switch, where I think you can basically accommodate both the binary intuition and the sort of continuous intuition. You know, if you have a light with a dimmer switch, like really, it is either on to some extent or it is not. And that is a real and meaningful difference. Either the circuit is open or the circuit is closed. With that being said, you can have, you know, electricity running through the circuit to greater or lesser extents. And that's also sort of a real thing.
This, I think, enables me to, you know, sound coherent when saying things like, it's really off for the table and it's really on for you. But I think it's more on for you than it is for a dog, than it is for a mouse, than it is for an ant. And so that's my own view. This is to some degree intuitive, again, because we don't have really strong grounding here, sort of just like giving you a dressed up vibe. But that is sort of my sense. I think it's fairly parsimonious. And the other thing I would say about this is I'm actually doing some work with Patrick Butlin right now at Ilios, trying to basically orationalize some of these indicators of consciousness. So this is sort of like what I was describing. We can look at these major theories of consciousness. They make specific predictions about what we would expect to see in systems that are conscious architecturally and functionally. And then we can literally just go in to a given system and evaluate whether or not those predictions are borne out. And so this is really hard to do with human experts. You got to get someone who's like, you want to do this with B-Cognition, you got to go find a B-Cognition expert, and then you got to explain to them what ignition events are in global workspace theory. And then you got to get them to... This just isn't a scalable approach for really evaluating the system. But my sort of grand innovation here is just throwing smart LLMs at this problem. And then being able to just scale the crap out of it so that we can evaluate, given any description of an architecture, of a nervous system architecture, biological, artificial, whatever, to what degree for each of these sort of indicator properties that are suggested by these consciousness theories, do we see those properties realized in these systems? And so we can actually go in and do this. And what you get out once you run this with a bunch of seeds, a bunch of different trials, a bunch of different judges checking each other, this sort of thing, are some really interesting implied probability numbers. I wouldn't say these are exactly implied probability that the system is conscious. It's maybe more like implied probability of consciousness relevant features given these theories. If you don't buy any of these theories, then like everything downstream of this doesn't really matter. But they're good, like it's like the best neuroscience has basically been able to do. You're aggregating across a bunch of different theories. There's a nice diversity there. And you get like really tight numbers across. So we have the best Gemini model, the best Claude model, and the best OpenAI model. And they all like basically agree. They agree 100% on the ordering of systems. So we do biological and artificial systems. And they sort of move around in terms of absolute scale. But in general, yeah, they rank these systems pretty coherently. And the reasoning is, as you might expect, pretty intelligent. And anyway, I mean, one punchline from that is the sort of implied probability of consciousness in something like a frontier LLM, according to these systems, is on the order of 30%.