Sourcery: Why Today’s AI Hardware May Already Be Hitting Its Physics Limit

Molly O'Shea (0:01)

When AI runs into the real world, the limits are no longer just about code... they are about physics. In this episode of Sourcery, Molly O'Shea speaks with Steven Latré from IMEC and Adam Chambers from Stripe about what comes after the GPU, and why today's AI hardware may already be hitting a wall. And the conversation opens with a pretty blunt assessment. Steven Latré explains that IMEC works at the earliest stages of chip design, and that the semiconductor industry is reaching the edge of what current approaches can do.

His quote really stuck with me: "We are essentially at the end of the physics of what we can still do." That's a bold statement. So what does that mean for Moore's Law? Well, Latré says the industry is "currently plateauing in what we call Moore's Law," after decades of shrinking transistors to make chips faster and more powerful. But now, with transistors down to roughly one nanometer, the next leap will require much more inventive hardware. Right, and Adam Chambers actually focuses on where the pain is showing up first. GPUs are still excellent for training, but inference is becoming constrained by power, energy, memory, cooling, and data transfer. He stresses that memory is already strained. What's the solution there?

Chambers says the next few years will depend on increasing bandwidth, reducing latency, and bringing liquid cooling closer to the GPU. And here's something interesting—the two also describe a hardware renaissance happening right now. How so?

Chips have strong margins, software is being squeezed by AI token costs, and demand is rising because there is simply not enough power or memory to go around. Chambers notes that 95 percent of the memory market sits with just three companies, which helps explain why prices are climbing. That market concentration is striking. But what about just building bigger models? Isn't that the typical solution? Actually, Chambers pushes back on that. He says the current wave of large language models still relies on brute force, and that many companies are now moving away from endlessly scaling model size. He adds that open-source models already exceed the human brain in parameter count, yet remain far less efficient. So if bigger isn't better, what's next? That's where photonics comes in, right? Exactly. Latré says optical components can move data faster and more efficiently than copper, without the same heat penalty. He expects co-packaged optics to move closer to the GPU within five years, helping solve the bottlenecks around energy, cooling, and data transfer.

And the episode closes with a strong prediction. Latré says, "In five to 10 years, we will laugh at how old-fashioned today's AI is," comparing present-day large language models to the dial-up era.

He also offers a founder tip: the best teams often combine European academic and technical depth with American business instincts.