Scaling your money safely with AI

Ryan Donovan (0:08)

Hello, and welcome to The Stack Overflow Podcast, a place to talk about all things software and technology. I'm your host, Ryan Donovan, and today we are recording live from the Ai4 conference. We're talking about, obviously, AI, but how you can keep it safe, trustworthy, not stealing money in a financial organization scale. And I guess for that is Srini Venkatesan, CTO of Paypal. So welcome to the show Srini.

Srini Venkatesan (0:36)

Nice to be here, Ryan.

Ryan Donovan (0:37)

We just got off stage from the keynote, talking about singer things. So I want to sort of double tap on some of the things we talked about. Obviously, Paypal has massive, massive scale.

It's like 400, 400, was it 300, I think?

Srini Venkatesan (0:51)

400 plus million customers.

Ryan Donovan (0:54)

Me too, we got my money.

But it's at scale and it's with money and you're introducing AI and some things. How do you think about the sort of safety versus like making, adding benefits?

Srini Venkatesan (1:07)

So it's a great question. Like we are in a very tight financial space. We move like trillions of dollars, as you know. So it's important for us that like, there are two aspects of AI that we worry about. Like one is like, how do we make sure it's secure and it's trustworthy? The second part is also ethical AI, right? And how do we use AI in a consistent way? There is no biases in what we do, right? So let me talk more about the first one, which is how we think about, how do we keep it secure and safe and so forth.

The key for us is validation more and more, right? As we go from code assist to loop engineering, to automated autonomous SCLC, the key is how do you validate it? How do you make sure that the checks and balances are there? So what we have done is we've focused more on the validation part of the equation, even more so than the coding part of it, to make sure that it is consistent and very clear. We still have human in the loop to make sure that the human is verifying everything that we do, and we can get that right all the time.

Ryan Donovan (2:12)

So you talk about validation, and usually that means sort of a LLM evaluation, right?

Srini Venkatesan (2:20)

It's both LLM validation and also actual functional validation. So the key for us is, so the challenge with the AI is, it's more general and LLMs are, it's more predictive. But what we do is we actually made them write deterministic code and then that is what we used to validate. Rather than like let the reasoning validate themselves, because then you get inconsistent results.

So what we try to do is like, yes, we have the reasoning out to create the deterministic piece, but where we want to be deterministic, we are trying to be deterministic. There's no point in predicting something which can be deterministically done. Ninety percent of our validation is our deterministic validations. We could have generated that using LLMs, but they are not runtime reasoning decisions.

Ryan Donovan (3:09)

Yeah, and that makes sense, especially when you're validating numbers.

Srini Venkatesan (3:13)

Yes.

Ryan Donovan (3:13)

LLMs are not graded numbers. Yes.

Something I've heard recently is using proving languages, like Lean or TLA+.

Srini Venkatesan (3:21)

Right.

Ryan Donovan (3:22)

Do you do any validation through that?

Srini Venkatesan (3:24)

Through some of these? Not right now.

The change is happening so fast. For me, it is about enterprise trade. How do we take every step, make it enterprise trade, get it out there for general consumption within my engineering population, and then go. We are leaning into stuff like foundational models and all that kind of stuff. But right now, the goal is to take it up a step level function within the company and keep progressing that way. Because getting 100 percent of two engineers is not as much as getting 20 percent more productivity from every engineer. And so enterprise scale is important for us.

Ryan Donovan (4:03)

Yeah, and that enterprise scale is tough. Because I think I've heard of folks starting with a five-person prototype, a testbed, and then scaling that up. Did you start with a small team?

Srini Venkatesan (4:18)

So what we did is we started slightly differently. We basically initially said we provided all three tools, Cloud, ChatGPT, and Poplaciti, to pretty much every employee and every engineer, so that they can get familiar with that. They chose to use our tool offering, whether it was Cloud Code, Turser, Codex, or Copilot to do code assist. So we get them to get familiarity with it. So every step was get everybody familiar with it. And then we started doing loop engineering with two or three teams. They developed expert systems and they created Autonomous STLC.