AI:AM: Where's the Steering Wheel? My Agent Set the Password, Hidden Valence, Proofs Find Linux Bugs

Nathan Labenz (0:00)

This week on AI in the AM, how do we give agents the access they need without letting our own control slip?

I do feel at the same time, some of this loss of control writ small already, where another thing that happened this weekend is for some reason, Claude decided it was in my best interest to rotate the passwords that are used in my agent console, and all of a sudden, I can't get into it. Now, I'm looking at my agent console, which has become the thing that I use all the time. I didn't set the password, it set the password. It can change the password on my access to itself. Kind of crazy. So I was able to resolve that pretty easily by just going over the terminal and starting a thread, and being like, hey, where did you put that password? I need it.

It wasn't actively trying to screw me over, so it just readily told me where to find the password, and I was back in business in two seconds. But you can see if the intent ever is bad on the part of Claude, it can already lock me out of my own shit. Cameron Berg, Director of Reciprocal Research, on a hidden valence signal inside language models that shapes their choices even while they say they feel nothing.

Cameron Berg (1:20)

There are clear alignment-relevant implications to there being internal states in these systems that aren't obviously reflected in the text that they output. What the model is saying and what the model is doing point at very different pictures of what's going on. And if we just sort of do the naïve text monitorability approach here, we would say, okay, well, the model's saying, you know, it can't say how this is going for it, and yet it's behaving as if it does have some sort of stake in how things are going for it.

Nathan Labenz (1:45)

Phil Maher, founder of StockFilm, an archive of 8-millimeter home movies from the 1930s to the 1980s, which AI companies want to license.

Phil Maher (1:54)

Yeah, I mean, it's been pretty popular. You know, we've talked to a lot of people, a lot of different AI companies, all over the world about licensing this out to them.

They want the original footage like that. They don't want anything cleaned up. They just want to take it for their own research and for training, of course, too. I mean, the 30s and 40s live alien. I like to say that the smile wasn't quite invented yet in a lot of the 30s footage.

Nathan Labenz (2:29)

On Tuesday, Google rolled out Gemini 4 widely. The question for me was whether to switch from the models I already trust.

So I had the opportunity to do a little bit of Gemini 4 pre-release testing. Why won't I swap the model immediately? It's because there's not enough of a step change to make me take any additional risk, and I do perceive there to be some. I'm not relentlessly performance maxing quite as much as I used to be. I'm now a little bit more trust and reliability maxing because the performance has got to such a point where it's really, really good. Everett Hildenbrandt, CEO of Runtime Verification, which mathematically proves software correct on where code is headed.

Everett Hildenbrandt (3:11)

And I also firmly believe that writing code is a dead art in two years tops, right? Like it's all going to be about writing specification and intent and making sure that, and so your package, part of your package delivery is not, not just, you know, did I get it into the registry correctly? Did I define the API? It's also going to be, did I provide the formal specs? Did I provide the lean fruits? Did I provide the actual logic that dictates how this package is supposed to behave? And I see no reason why that will not be the case in five years tops, right? There may, you know, we're going to have this intermediate period between now and then where it won't be the case, but I think gone are the days of, of what I think of as like the cowboy coder or something like that, right? Like someone who is just like so good at making software that they're just going to, you know, go and make a bunch of software that's going to change the world or something like that. You're not going to have artisanal software anymore. We're going to have past manufactured software that, if it doesn't fit the mold of, it's been specified, it has the correct API boundaries, it has formal verification attached to it, it doesn't fit in the package repository, right? Because it just will be too risky.