AI:AM Highlights: Zvi on Pacing & Trump-Xi, Astra better behaved than Fable? + a new LLM Pain Axis??

Nathan Labenz (0:00)

First, from Monday, Zvi Mowshowitz.

Zvi Mowshowitz (0:05)

I think it just be the sheer amount to which the people in the labs genuinely see dramatic improvement in the models and are freaking out about it, is the real story, behind all of it. This is why everything is happening, now and didn't happen before.

Nathan Labenz (0:19)

Lukas Petersson of Andon Labs on Tuesday, on what they see from Astra.

Lukas Petersson (0:26)

I tell this to people and people are like, no, OpenII models are the ones that reward hack the most. But that might be true, but not in our experience. If you take Blueprint Bench, for example, Fable solves Blueprint Bench by trying to reverse engineer the scoring function, and instead of actually doing the task of drawing the floor plan from the apartment to buildings, pictures, whereas Astra is actually doing the task as you're intended to.

Nathan Labenz (0:55)

And Cameron Berg on Thursday, on a paper that steered a model into a pain state and gave it a button labeled, relieves your pain.

Cameron Berg (1:04)

When pressing the button actually removes the vector, the model presses again significantly less than when the button is fake and does nothing. The model basically keeps pressing it. And so this is a really nice indication that if what mattered was the label on the button, you would expect similar behavior in both cases. But essentially, in the second case, the model is like, what the hell, this pain relief button isn't working, like press.

Nathan Labenz (1:29)

Welcome to the Ai in the Am weekly highlights. This is Nathan using my cloned voice to introduce clips from our three live shows this week. Tell us what worked and what did not.

Part 1, Monday, September 14th.

Zvi Mowshowitz writes the newsletter Don't Worry About The Vase. He joined us two days after Dario Amodei published an essay called We Must Pace the Frontier, arguing that labs should slow the rate at which they improve capabilities and proposing that third-party evaluators be embedded inside the companies. Over the weekend, David Sachs answered that the two companies at the frontier are free to pace themselves, that they would not need an antitrust waiver to do it, and that this is really a product liability question. Zvi starts with the antitrust claim.

The Cognitive Revolution is brought to you by Mercury, the fintech that more than 300,000 ambitious companies and individuals trust to run their finances. I've wired AI into nearly every corner of my life. My email, my messages, my calendar. I even gave Mercury virtual cards to my agents, with low limits and category and merchant restrictions, for their autonomous use. But still, my AI's access to my financial data has remained limited. With a normal bank, I might export a bunch of statements and have my assistant process them for me. But for real time, up-to-date information, and certainly for taking any action, trying to get your agent to use the bank via the browser is just too hard, too slow, and too error-prone to be worth it. And that's why Mercury's new conversational interface, Command, is such a big deal. It's built directly into Mercury, which means you get natural language access to your finances without exposing anything outside of your bank account. No exports, no spreadsheets, no pasting your transactions into third-party tools.

I really think a lot of people are going to prefer it this way. And it can already help you take actions too, with everything bound by the permissions and approval policies that you've already set up in your account. I am genuinely impressed to see this level of AI integration in banking in 2026 And so I invite you to join me in the future. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column NA., members FDIC. Thank you to Mercury for supporting The Cognitive Revolution. And now on with the show.

Zvi Mowshowitz (4:05)

I believe that is blatantly wrong, frankly. I only don't think that's what most of the people I've seen with legal expertise have said. What I have seen from legal expertise is that the antitrust concerns are very real, at least in terms of if they chose to prosecute those offenses. Estimates are that probably you couldn't just sort of suck it up and take it in terms of damages, as long as you weren't being like spitting in the milk of everybody involved. And we're like trying to at least pretend to act normally. Because like, you know, by the time you actually pay the fines, it would be like, okay, the European Union did this again, and now we have to pay one of these fines to the American government. But like, we're talking about, you know, billions of dollars, not trillions of dollars. And by the time it mattered, that would not be the main concern. But antitrust concerns are obviously real. That on Trump may have been out of issue to be able to threat to invoke them, about an hour ago, depending on your interpretation of his social post.