Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO

**Swyx** (0:04)
We're here with Akshat of Modal, CTO of Modal, together with Vibhu. Congrats on your 3C.

**Akshat Bubna** (0:10)
Thank you.

**Swyx** (0:12)
Your party yesterday was amazing.

**Vibhu** (0:15)
Yeah.

**Swyx** (0:15)
All the photos and all the swag.

**Akshat Bubna** (0:18)
We had a bunch of art installations, which is kind of fun seeing our products on pedestals next to Rodin.

**Swyx** (0:26)
Very nice.
When you started, it was not the GPU inference company. I mean, maybe it was in your mind. Take us back to the origin story.

**Akshat Bubna** (0:39)
I actually first met Erik, who's the CEO, through an investor. And back then, Erik was already thinking about building a new kind of runtime. And he got there thinking through why are workflow orchestration products so hard to use. It's because you have to run them on Kubernetes. Kubernetes is hard to manage. It's not built for burstiness and custom images. It has a terrible developer experience.

**Swyx** (1:05)
And I'll interject for listeners who are new.
We interviewed Erik two years ago. And there's a bit more of the story there from Spotify and all those things. And I actually came across Erik through Data Council because he did that talk on the serverless container stack that you guys did, which was like, that was my first like, okay, I need to take Modal's very seriously moment. But it's still very unclear, like, do I actually need all this for just my data pipelines?

**Akshat Bubna** (1:33)
Yeah, I mean, initially what we were thinking about was if we build a better runtime, it's a very useful primitive in itself.
There's a lot of things that get solved by serverless functions, like you can do ETL stuff, you can do job queues, you can do all this like bursty processing, which it turns out every company had needs for. But then we also were thinking about this as like, this is a primitive that we can build a whole collection of products on, which are very verticalized. So perhaps data engineering would have been the first one, but we were thinking about inference. Back then it was more classical inference, like computer vision stuff and running XGBoost and whatnot. But we added GPUs to the product a year before ChatGPT came out. We just didn't think it would be that big of a deal.

**Swyx** (2:21)
Yeah, just like add A100.

**Vibhu** (2:23)
Was there any early key problem that really sparked off why you built it?

**Akshat Bubna** (2:28)
Yeah, primarily it's just, not only is the tooling that was out there was built for, one, a really great developer experience, and also there's a general trend of, a lot of the workloads that we were seeing were very, I wish there was a better word for it, but compute heavy.
One, need a lot more resources, so you need to burst up and down a lot, versus Kubernetes is designed for slow scaling, and more for web server use cases. And also, there's just a lot more specialization in what kinds of environments these workloads run in. Sometimes they need accelerators, sometimes they need different kinds of images, and this is a consistent thing that we saw across a lot of companies. That would be the next step.

**Swyx** (3:14)
I don't know how much this factored into the early story, but I wrote a post when I was at Temporal about infrastructure, software-defined infrastructure, something like that.

**Akshat Bubna** (3:22)
The self-revisioning website.

**Swyx** (3:23)
Yeah, I can't even remember my own post. And then you put me on the landing page.

**Akshat Bubna** (3:28)
Yeah, we really liked the term, and so we stole it.

**Swyx** (3:32)
Because you had the insight that everything can just be in decorators, next, co-located with the code, right? Was that a big part of the original story, or it was just like a DX layer?

**Akshat Bubna** (3:41)
That was really important because we really didn't want people to spend so much time writing YAML, and it seemed like you could really condense the surface area of what you're doing, put it in code so you can actually operate on it, just like you can operate on other code, and build stuff that's more expressive and dynamic. And so yeah, that was always a very important part.

**Swyx** (4:04)
Then the pushback is, this is a DSO.
It's your closed source. I am locked in to Modal.

**Akshat Bubna** (4:11)
Yeah, we never really got pushback for that because the nice thing about Modal is you can bring whatever code you have. And sure, the DSL is at the sort of configuration there for what hardware you're using, how you're scaling things up, but you still own the code.