Building Production Workflows for AI Applications

**Tony Holdstock-Brown** (0:00)
Products, the table stakes have gotten higher, they're harder to build, especially post-AI. People just expect more. That means that if they expect more, you've got more to do as an engineer, as a developer, and you should not necessarily be concerned with mucking about with low-level queuing infrastructure, setting up dead-letter queues, figuring out how to retry subservability and the rest, and also managing state on top of that. And so there's a whole bunch of developer tools, infrastructure tools and APIs that are being created so that developers can be more effective.
And AI does something similar, in which it makes developers more effective at problem-solving. As long as you can understand the application of what the LLM is saying, and don't take it for granted.

**Derrick Harris** (0:39)
Welcome once again to the a16z AI podcast, where we cover the fast-moving AI landscape from any number of different angles. I'm Derek Harris, and this week, we're discussing the challenges and requirements of running AI applications, including LLM-based agents, in production. These include things like ensuring workflows execute how they're supposed to, that limited GPU resources are allocated fairly, and even using generative AI to help build transactional applications and workflows in the first place. Joining me to break it down are a16z partner, Yoko Lee, and Inngest co-founder and CEO, Tony Holdstock-Brown. Enjoy.
As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Tony Holdstock-Brown** (1:42)
I taught myself how to code when I was like 10 or 12 I go into like PHP, Java, Visual Basic 6, and one up back in the day when I was a kid, and super fun.
Kind of picked it up from there, ran with it. Technology since I was a kid. Startups since I was in my late teens, early 20s.
Eventually went on to run engineering for a healthcare company, which is where Inngest comes from. Healthcare is really, really, really difficult. I respect everyone that works in it. Super hard, don't love it anymore. Really, really, really challenging, and you need to build a lot of pipelines, which is super relevant to AI actually, because a lot of things are flows. Do this, then do that, then do this, and if this happens, do X, Y, and Z, which is sort of similar to how people nowadays build LLM pipelines and all these agentic workflows.

**Yoko Li** (2:26)
What's the first thing that you coded up when you were learning to code?

**Tony Holdstock-Brown** (2:30)
I remember making this really, really terrible hacking game. I remember making this really bad joke machine.
Dumb things, games, stupid apps, you know, that sort of stuff back in the day.

**Derrick Harris** (2:41)
When did that transition into building real things?

**Tony Holdstock-Brown** (2:44)
I don't know, you end up building all these things for fun, exploring the problems, like learning, and I think there's something really, really interesting about building something that in a way is tangible, you know, like you write all this code and then you actually have this usable thing. And that's really gratifying, especially as a kid when you're exploring technology. But after a while, if you build things and you're like, oh, cool, yeah, I've built it, like, yay, that sort of novelty wears off and you want to build things that other people use.
So that's how I sort of started getting into things on the internet, because like when you publish things on the internet, when you publish like games on the internet, when you build things that are usable, other people give you feedback, other people comment on things. So I think like the slow transition was wanting people to use the stuff that you've built. And then after that, you end up just, you know, getting a job at a startup so that you can actually do this full time.

**Derrick Harris** (3:34)
I'm curious to what your experience was, or maybe with the role that like your time at Docker played, because you were at Docker during the height of like 2015, seemed like peak Docker maybe. So I'm curious how that influenced what you've done since, because it was obviously like kind of a big deal.

**Tony Holdstock-Brown** (3:50)
Yeah, kind of a big deal, a wild time. And firstly, super grateful to work there, because the team were, I guess, a lot of people ended up moving on. Team were fantastic, learned a lot. A lot of them are still really close friends.