20VC: Harrison Chase on Inference Demand, Routing, and Why Agents Break

SPEAKER_1 (0:01)

When demand collides with complexity, the winners are often the teams that can turn chaos into systems.

Today we're unpacking Harry Stebbings' conversation with Harrison Chase, founder of LangChain, about the future of inference and why building agents is so damn hard.

Chase opens with a reality check that caught my attention: "I really underestimated the demand for inference." The market's far bigger than anyone expected, and even as models improve, usage keeps climbing because the work is spreading into more products and workflows.

Right, and LangChain sits right at the center of that. It's essentially developer tools for agents—the open-source framework connects models to tools and data, while LangSmith handles observability and evals. His core message? Agents fail constantly, so builders need visibility into where they break. That's the key challenge. Chase says the real problem is non-determinism. Models are black boxes that take natural language in and out, and agents aren't just one call—they're many calls chained together.

One small error cascades into a much bigger failure, which changes how you build products entirely. Chase says teams often have to "delete the product" as models improve, because yesterday's workaround becomes unnecessary. The right harness changes every few months, so you have to distinguish between short-term hacks and long-term necessities.

Building on that, he's really focused on portability. Chase argues for neutral harnesses because agents accumulate memory, context, and state over time. If that intelligence gets tied to one model provider, you're locked in. He calls the alternative "owning your intelligence." That logic extends to vertical agents, where he's especially bullish. The real value comes from domain experts who can build strong evals and verifiers. Those teams know the domain well enough to engineer the right context and harness before even touching the model. Interesting point about model choice too—most companies aren't training from scratch. They're post-training open-weight models after exhausting gains from context and harness optimization. But Chase doesn't think vertical builders should start there. He sees post-training as useful mainly for simpler tasks. Then there's routing, which Chase says is definitely not a commodity. The smartest routing is harness-aware, because the harness knows the task, domain, and state. For vertical agents, routing belongs inside the harness; for broad chat or coding, a gateway might be enough. He's cautious about evals too. Academic benchmarks become stale quickly, while internal domain evals remain valuable. But even then, teams should study traces and data directly, not just trust a single score. On the economics side, inference demand and coding-agent usage are still climbing. Organizations need cost visibility, budget controls, and smarter harnesses. Prompt caching and intelligent routing help, but the broader trend is still upward.

By the end, Chase is clear about where he'd invest: inference, lower-level infrastructure, and vertical agents with real domain knowledge. His final message is practical—build fast, stay close to the data, and expect the stack to keep changing around you.