Best practices for scaling AI adoption across a large org & the trade offs you will have to make w/ Sagnik Nandy #273

SPEAKER_1 (0:02)

The CAFE's framework is a definition framework for what constitutes high quality context. The first dimension C is for clarity. Clarity asks whether the agent can interpret the request the way that it was intended. A request can reasonably be read more than one way. So that is something that can feel obvious to the person, right? It can feel obvious to us because we wrote it, and we bring everything that we know into reading it. Say you ask an agent to clean up the old feature flags. The agent is going to read old, let's say as something that has not been touched in six months across the entire repository. So it will go and remove the flags in other Teams services too.

One of them might be a kill switch, which is supposed to sit there and be untouched until the day you need it. But because the first step was wrong, everything after it builds on it. So one ambiguous line can turn then into hours of very confident but wrong work.

Jerry Li (1:05)

Stay tuned for a little bit in this episode to hear more from our friends and sponsor DX on their new Cafe's framework and follow the link in our show notes to download the paper for yourself.

Sagnik Nandy (1:17)

I think this is now part of all of our jobs, and that's an important thing. If this is treated as an aside, separate thing, it'll show in how we function. I feel we're in a new world. Right now, for example, video conferencing tools are a regular part of how teams work. Twenty years back, they were less so. So now these AI tools are part of the regular functioning of a team. It's not an exception. This is the new norm. Leaders have to manage these tools and manage the deployment of these tools and the usage of these tools in specific ways. So we've made it clear that this is part of your job. We've made it clear to the organization, like it's not optional. We expect you to use it. If you get it, not everything will be right. But at the end of the day, we are all responsible for leaning in. And this is now part of our jobs. This is not a nice to have. This is not like a good 5%.

And you get a check and say, yes, no, we will regularly check usage, productivity boost, everything.

Patrick Gallagher (2:12)

Hello and welcome to The Engineering Leadership Podcast brought to you by ELC, the engineering leadership community.

Jerry Li (2:19)

I'm Jerry Li, founder of ELC.

Patrick Gallagher (2:21)

And I'm Patrick Gallagher, and we're your hosts. Our show shares the most critical perspectives, habits and examples of great software engineering leaders to help evolve leadership in the tech industry.

Jerry Li (2:34)

I'm Sagnik. I'm really looking forward to our conversation. We chatted a little bit about the information and how you were able to increase the adoption across the company. There's a lot more topic I want to dig into today. So before we start, would you mind telling us more about yourself and your role and why you're excited about this conversation?

Sagnik Nandy (2:52)

So my name is Sagnik. I'm the CTO at DocuSign. I've been here for two years. Prior to that, I spent three years at Okta. I was the Chief Development Officer running product and engineering there. Prior to that, I spent 15 plus years at Google. I was a VP doing all sorts of things in the ads, analytics, data. The last two years at DocuSign have been phenomenal. We've both been building our AI first agreement platform, IAM, Intelligent Agreement Management. That's the platform.

It's a whole new set of product functionality that really helps streamline the agreement journey, leveraging AI. So that's been amazing. And as we are building it, we've also become increasingly AI adopting, AI leveraging ourselves, and often putting ourselves in the position of our customers. When they are thinking of AI tools, how are they thinking of it? We want to do the same thing internally, lean into AI tools, do things in an AI-first way. And that whole journey of both leveraging AI in the engineering development process, as well as building AI-first products, that combination has been very fulfilling.

Jerry Li (4:01)

How many engineers are there at Docusign? Let's just give us a context of the scale of this transformation.

Sagnik Nandy (4:07)

Yeah, it's a thousand plus.

Jerry Li (4:09)

And how do you approach this problem? Because it's a new tool, new technology, and trends manage have been one of the hardest leadership challenges across all organizations.

Sagnik Nandy (4:21)

Yeah, so at the very beginning, around a year and a half back, when we really, as a group, felt that we'll go all in, we made a few choices. We said we would focus on value. So instead of gimmicks or one-off examples or one good anecdote, we said we'll focus on value. So a lot of measurement, a lot of agreeing on what good looks like and moving up the value chain. That was one thing. The second thing we decided was we will build the right guardrails. So you have to build the right framework before you can run. We would not nickel and dime in building the right framework. So a lot of investments and focus on trust compliance, building the right guardrails so we can then keep moving fast. So that was done. The third one is around sustainability. We saw that we were calm during all the token maxing, all of that. For us, we want to do this long term. So again, less for gimmick, more for value. But sustainability is important. So getting good throughput, doing it right, not making it unsustainable for the developers, understanding what really good means. That was the third thing. And then finally, we wanted to scale our goodness.