You need reliable AI context for your site reliability

**Ryan Donovan** (0:07)
Hello, and welcome to The Stack Overflow Podcast, a place to talk all things software and technology.
My name is Ryan Donovan, and today we're talking about context engineering in terms of AISRE. My guest for that is Asaf Savich, AI engineering group manager at Komodor. So welcome to the show, Asaf.

**Asaf Savich** (0:28)
Hello, hey Ryan, thanks for having me.

**Ryan Donovan** (0:30)
Of course. So before we get into the context engineering discussion, tell us a little bit about yourself. How did you get into software and technology?

**Asaf Savich** (0:37)
So I wanted actually to be an accountant. That's what I started off with. Yeah, I like numbers. I like money. I thought that was a good combination. And this is where I wanted to be. But then I was looking for a job and like a QA opportunity came in my way and I started doing that. And I just fell in love with software and just felt right. It felt the natural thing for me.
So I went to school and started like a computer science, and it made me even more fall in love with the topic. And during my studies, I started working as a software engineer part-time in a few small startups. And then I also understood the startups are also the right fit for me. So that was my beginnings.

**Ryan Donovan** (1:20)
The software industry as a whole is sort of feeling the effects of small decisions as they push Vibe-coded code to production. And I think a lot of folks are looking at ways to do the DevOps operations management. So let's talk a little bit about using AI in SREs. We've talked to a few folks who do that sort of thing. Code and production is a big topic, right?
What does SRE cover?

**Asaf Savich** (1:48)
So let's start with what an SRE actually is.
Today, in many organizations, SRE is a very pivotal position. SRE stands for Site Reliability Engineer. Personally, the guys who are responsible for keeping the production environment stable, healthy, and to make everyone's happy, right? Both the company providing the service and the users who are actually consuming the service.
So SREs have that job and it's a lot of responsibility to do that job and their part includes so much context to it, much before the AI era. It includes so much context to it, to understand the business side of things, where those things fit, what's the relationship between services one to another. And over the years where we added a lot of capabilities of microservices and a lot of networking layer and cloud infrastructure. So we've just kept adding a lot and a lot on the shoulders of the SREs, we just have to make sure that production is actually reliable. So we put a lot of effort and a lot of responsibilities over the shoulder and just keeping adding those is just a bit problematic. So AI is really good at helping the SREs and making them be much better.

**Ryan Donovan** (3:00)
That's a great point that as people have created microservices as a way to organize teams, SREs have to know about all of that. That is a lot of information to have all at once.
With AI agents, one of the big topics now is context. How do you get context for all these AI decisions? And we're just saying there's a lot of context for SREs in general. How do you get that context into an AI and have it be effective?

**Asaf Savich** (3:28)
So we started off in a very naive way of just throwing a lot of stuff at AI and start to understand what the agent is capable and is not capable of doing.
And really fast, we got to a point where it was just too problematic. And the data we had to inject into the AI agent, we had to think about it really hard and really long on like, what's the proper way of injecting the data? What kind of information is more valuable than others? So we had to do a lot of stuff. So we focused on two main areas. One is actually the data and how to collect and to inject it. And the second is the evaluation layer that no matter what change do we think is important or seems the right way to go.
We have an evaluation layer, which each change that we are conducting, we want to make sure that the change is in fact, putting us in the direction that we want to head up. So these two must live together, one without the other is just not enough. And it's not enough for every AI agent, especially AI agents that claim to be the ones that are actually resolving production issues and putting us in the most sensitive place in the organization. So we have to balance between the two and work on two on a parallel way.