Lessons Learned from Programming at Google - Part II

**Dan Lines** (0:00)
Google, so big, they deserve two episodes.

**SPEAKER_2** (0:04)
Last week, we had two of Google's top engineering leaders, Hyrum Wright and Titus Winters. And this week, they're back for part two, talking about their fantastic book, Software Engineering at Google, Lessons Learned from Programming Over Time.

**SPEAKER_3** (0:22)
This episode is sponsored by Linear B. Give your dev team the power to improve with team-based metrics, high-risk code alerts, and the world's first project board based on real-time Git activities.
Sign up free at linearb.io.

**SPEAKER_2** (0:37)
Hey, everyone.

**Dan Lines** (0:38)
Welcome to Dev Interrupted. I'm your host, Dan Lines, and today we are talking to Hyrum Wright and Titus Winters.

**SPEAKER_2** (0:45)
Hyrum and Titus, welcome back.

**Titus Winters** (0:48)
Thanks for having us.

**Dan Lines** (0:48)
Great to be here.
This has been a really insightful, interesting, amazing conversation. Are there any other areas within Google that shifting left is working well? And also, is there any areas where you would say, hey, shifting left is actually a bad thing?

**Titus Winters** (1:06)
So amusingly, I just hosted an internal workshop discussing the left shifting philosophy just recently. And there were a couple of points where we realized, yeah, you can go to something like, if you have editor save hooks that go through and delete your unused imports, that might be annoying. I wrote that import, I'm about to go use it. I accidentally had saved and you just deleted the thing. Probably don't do that. That is shifting the fix a little bit too aggressively to the left. But there are a lot of things where we're finding deeper value in having pushed it this way. Like one of my favorites, and it's alluded to, but even my thinking on it was not as advanced at the point that the book was finalized. One of my favorites is a couple of pages in the continuous integration chapter, where I'm gonna argue that everything that we know about monitoring and alerting in production is directly parallel to unit testing and CI.
All of the stats, all of the things that you're observing, QPS, disk usage network per application statistics, everything that you're using that you're monitoring in production, the same logic and philosophy applies to unit tests.
And your CI is exactly the same as your alerting system. And therefore, you can go read all of the SRE knowledge, wisdom, philosophy on monitoring and alerting and apply that to your testing and level up immediately. And that's brilliant. So yeah, there's a lot, we're not done yet.

**Dan Lines** (2:35)
Great concept there, totally makes sense. Let's discuss trade-off. Hyrum, at a high level, it feels like maybe every decision even in life, there's some sort of trade-off that maybe needs to be made. How should we think about it in relation to software engineering?

**Hyrum Wright** (2:52)
I had a professor in grad school who said engineering is the science of trade-offs. This was in electrical engineering or some, much more physically constrained, if you will, engineering discipline. But it's the science of trade-offs. We have to make decisions about, are we going to put more engineer time on a specific project to try to optimize it and perhaps save computational resources? Are we just going to throw more computational resources out of problem and not worry about our software? What are we going to choose to spend our time on as we are software engineers, so we're doing things? One of the discussions that Titus and I have been involved in a lot internally is around technical debt. And that domain, how do we address technical debt internally? How do we incentivize it? It actually turns out that a lot of people are really interested in cleaning up technical debt. It also turns out there's a lot of technical debt that you don't need to clean up, that it's an acceptable level of technical debt in kind of the ecosystem. I hold another talk, you can go find where I talk about like technical debt as pollution in an ecosystem, because there's a much more varied kind of way of thinking about it.
But people say, hey, I cleaned up this thing, I spent my whole week doing this, and we sit back and say that wasn't actually the most important thing for you to do. You just traded off your time for a thing that wasn't actually that valuable. So in the software engineering discipline, we need to be thinking about, number one, what are the most impactful things that we can be doing?
Then how do we make those decisions in our organization?
We try to use as much data as we possibly can. If we could make every decision, if we had perfect data, we wouldn't actually need to make the decision, because the decision made for us, we have an automatic system that makes the decision. The reason we pay people and humans to do things is because they need judgment and they need insight, and we don't have perfect information. We have to make decisions underneath those constraints and move forward. But we also need to be willing to revisit those decisions as the constraints change. So as the system evolves, as the data changes, as our ability to do things changes, we need to be able to revisit those decisions. Several years ago, when we first started doing these mass migrations within our codebase, we obviously looked at the most highest priority ones. What are the things that really matter the most? And let's focus on that. And as we got better and better at this, one day Titus came to me and said, there is a file that has a dash in its name.