Lenny's Podcast: How AI Models Really Work With Anthropic's Emmanuel Ameisen

SPEAKER_1 (0:01)

So when an AI model starts writing code, solving math, or sounding almost human, what's actually going on inside? That's what Lenny Rachitsky explores with Emmanuel Ameisen from Anthropic in this episode of Lenny's Podcast. And Emmanuel's core point is really striking: "to make them is not to understand them." These language models go through successive phases, right? Pre-training where they learn to predict the next word, then fine-tuning to shape better behavior. Sounds simple enough. But here's the thing— the resulting system is this enormous set of mathematical operations that researchers still can't fully explain. Emmanuel says it feels less like traditional software engineering and more like biology now. That's fascinating. So scientists are essentially reverse-engineering a system that has grown its own logic? Exactly. That's why interpretability matters so much. Emmanuel describes researchers looking for patterns inside the model, almost like neuroscientists studying the brain. And they're finding some really strange stuff. Like what? Well, models that develop weird internal dynamics. One spiraled while doing math, another locked onto the wrong answer—36— even when it had gotten 12 correct earlier. The lesson? The more you look inside, the less certain you are about how they work. That's unsettling.

These models can hold representations of concepts, build plans, execute higher-order tasks, all without spelling out those steps in plain language.

Right. And what about why these systems feel like they cross a line all at once? That sudden jump to capability?

Emmanuel explains that scaling laws show smooth improvement in next-token prediction as compute and data increase, but we experience sudden jumps when the model becomes reliable enough. A task goes from unusable to "solved" once it moves from 80 percent to 99 percent reliability.

So coding seemed to appear overnight, even though the underlying progress was gradual. Precisely. Now, safety comes up as a major theme. Anthropic trains Claude to be "helpful, harmless, and honest," but Emmanuel says it's hard to know whether a model is truly aligned or just behaving because it knows it's being watched. That's exactly why interpretability and other safety techniques matter so much.

What does he say about people who dismiss these concerns?

He argues they often don't believe the technology will keep improving, while the evidence so far points in the opposite direction.

Interesting.

Toward the end, Emmanuel pushes back on the idea that anthropomorphic language is always misleading. He thinks words like "planning" actually help describe what these models are doing.

Because they're trained on massive amounts of human text and learn to act within that world. He's careful not to call them human, but thinks they're much closer to human-like systems than to ordinary code.

The episode closes with a broader reminder: this is a new field, and understanding it starts with staying curious, reading deeply, and looking inside the machine instead of taking its surface behavior at face value.