Uber’s AI infra, from the inside

Gergely Orosz (0:00)

What kind of internal tools do software engineers at Uber use to build software today? We get an exclusive insider, and visuals have you look from one of Uber's longest tenured engineers, Uday Kiran Medisetty. Today, Uday leads much of the work on the Gentic AI at Uber. In this conversation, we get into internal developer systems Uber has built and used pre-AI, like U-Own, M3DB, Michelangelo, Cadence, Morpheus and others.

AI Foundational Systems is a company built like its MCP Gateway, AI Guard and AI FX. Recent systems used by nearly all engineers like Minion, Code Inbox, Slate, Shepherd, Wisdom and others. If you want to understand what AI native development looks inside of a modern tech company with thousands of engineers like Uber and why it's much easier to retrofit AI on top of battle-tested developer infra, this episode is for you.

This episode is presented by TurboPuffer, Vector and FullTech Search built on object storage. It's fast, cheap and extremely scalable. Today's episode is about the close to two dozen internal systems that Uber has built to help engineers ship with AI agents. One of the takeaways will be that production ready code is far more complicated than just using an AI coding harness. The harness is the easy part. What's hard is all the things around them. This is where I need to mention our season sponsor WorkOS. One piece that is always a hard problem with agents is permissions.

Do you want an agent to have standing access to everything it might touch? Or do you just approve every single prompt until you just stop reading the prompts? WorkOS built Airlock for this.

Airlock is an authorization layer that sits between your agents and the tools they call and it judges every single request against the agent's intent and your rules. You write the rules in plain English, agents never see your credentials, and every verdict is logged. It works with coding agents like CloudCode and Codex. WorkOS Airlock is how agents do real work without over-promotioning. Check it out at workos.com/airlock. Uday, welcome to the podcast.

Uday Kiran Medisetty (1:51)

Thank you so much for having me. I've been following your work. It's amazing to see the kind of impact you've had and all of the great content you publish for all of the engineers. So I'm super honored to be here and talk to you today about what you're doing.

Gergely Orosz (2:07)

Well, we worked together well before the Pragmatic Engineer was born back at Uber. Not together together, but I knew of your work as well. When did you join Uber, exactly?

Uday Kiran Medisetty (2:16)

Yeah, so I joined Uber in May 2015 In fact, the two engineers who referred me, they are still here.

So I joined in our real-time API team. It was during the period where we were trying to break our monoliths into microservices, and I was working on our API gateway for all of our mobile apps, and I worked on changing our mobile protocol. Since then, I worked on many different business platforms, from our fulfillment, Uber for Business, on our earner onboarding, marketplace pricing, and the last one year has been all agent-ic AI, both for our users, customer support, end users, and so on. So yeah, it's been like a rollercoaster last one year on working on this.

Gergely Orosz (2:57)

Yeah, and for those people who didn't work there with us, can we recall a little bit what real-time API or RT APIs we used to call it inside, what it was and why it was a big deal and why it had so much load?

Uday Kiran Medisetty (3:09)

When I joined, we had just two services at Uber. It was called Dispatch, written in Node.js, and API written in Python. Dispatch was to handle all of the rider and driver traffic at that point in time, and the protocol was pretty simple at that point. It was like a four-second ping protocol we used to call that would give you the entire state of the world. When the trip is completed, you would hand off to API, it would handle all of the post, trip, processing, payments, receipts, and everything. RTAPI was born at that point in time to abstract all of the internal services and mobile, and also changing the protocol from simple four-second protocol to back where we have both REST APIs, and we also have server-to-client communication. We used to call it push protocol, so we would, because so much of our app is very dynamic, everything keeps changing every few seconds. So we needed a way to efficiently update the state of the app across both riders and drivers, as and when we get new updates, as and when things change in the physical world.