Is 99.9% safe good enough for robotaxis? | Phil Koopman of Carnegie Mellon University (Professor Emeritus)

John Heinlein (0:00)

Today in The Garage, we're going to look at autonomous driving in a new way. Everyone is excited to see how much progress autonomous, vehicles have made in recent years with Waymo and Zoox and many other companies now commercially available. Many people in in cities like here in San Jose, the Bay Area where we live, can hail a robotaxi on their phone and it shows up with no driver and takes them on a ride. And it's been incredibly exciting and successful to see. I, personally, in my career was working on robotaxis about five years ago when it wasn't quite ready and it was a very frustrating experience for me. So to see how much progress they've made is really inspiring to see how that technology is going and an incredible use of AI. Having said that, as we're getting to higher numbers of robotaxis, the complexity of making robotaxis that work well to make them at the level of working well that is viable to scale to thousands and thousands of vehicles is a bigger challenge. And who best to talk about that than a real expert who's been studying this for thirty years? Our guest today is professor, now newly retired, Phil Koopman from Carnegie Mellon University who spent thirty years studying robotaxis and other aspects of safety engineering and will tell us all about the pluses and the challenges to make robotaxis very viable in scale.

Let's go.

I'm John Heinlein, Chief Marketing Officer with Sonatus. We're so pleased to have Phil Koopman here with us today. Phil, welcome to The Garage. Thanks for having me on. Happy to be here. So we we both have a strong connection to Carnegie Mellon. I went to undergrad there. You were a professor for a long time. And we we I think we crossed paths when back at when I was a student and you were a grad student. Start by telling us your background. Sure. My background is I started with an undergrad in computer engineering at RPI.

And after that, I served in the Navy because they were kind enough to pay for my undergrad. That that's how it worked back then. I then worked well, then I went to Carnegie Mellon University, got a PhD, which is where we ran into each other. I was a teaching assistant in a hardware design course, bit-slice CPUs for people who know even what that is.

And but it was a fun course. And after I graduated there, I went to work for Harris Semiconductor as a CPU designer. I spent time at United Technologies at the Central Research Center, got a lot of embedded domain experience, and then finally went back to Carnegie Mellon as a professor, taught there for almost thirty years, and just recently retired. Well, congratulations on an incredible career, and you have a real distinguished background. But we're gonna get to that. But before you, you gotta tell us a fun fact about you. Sure. The the fun fact is I used to drive submarines in the navy during the cold war. I have a combat medal, but I can't tell you how I got it. That's an incredible story. I love submarine movies, Das Boot and The Hunt for Red October. Such such great stories, so I'm jealous of your experience. Well, Hunt for Red October is an exceptionally good one. It's very accurate for the types of things I was doing back when I was doing it. Very fun. So tell us about your career at CMU and especially about safety engineering. That was a key specialty you had. I wonder if you can introduce for our audience, what is the concept of safety engineering and how does it differ in some ways from sort of, if you will, classic engineering of of just building something? How does it complement that? Sure. I've been doing self- driving car safety specifically for thirty years, and I've been doing a lot of other types of safety engineering for I lost count at 200 design reviews of embedded systems, and I lost count like 15 ago.

And and some of them were just doesn't work, but a lot of them were safety in all sorts of different domains. So I've seen lots of safety standards, lots of safety. And if you have to boil safety down, it really comes down to if you're building a system and you're testing it, engineers are taught to make sure it works. And they're really good at the happy case, but maybe not the unhappy case.

And safety engineering isn't about how to make it work. It's about what happens when something goes wrong. And specifically, what you're taught to do is create a list of hazards, things that can cause problems. A lot of times, it's personal injury, but it could be severe equipment damage. It could be a lot of things. It's something bad will happen. You make the list of here are all the things that can that can go wrong and here's our plan to make sure that either it won't happen or if it does happen, that the harm will be mitigated somehow. And so safety engineering is listing the hazards, figuring out how to mitigate the risk from the hazards, and making sure those safety requirements go into the engineering requirements, which means now you have to design the safety behavior intentionally and you have to test that the safety stuff really works. Just a really simple one: On a train, there's air brakes. If everything goes wrong, there's a cylinder of compressed air and you hit the button and it slams on the brakes. And and what's that for? Well, there's lots of things that you go on a train, but most of them get cured by slamming on the brakes. So that's the plan. It's interesting to to describe it that way and I think that, you know, as you're right. As an engineer, you want to make something work, but, know, you have to really think about all the failure scenarios. And I think there's a concept called fail safe and fail dangerous, which I think is is often often talked about. I think probably your goal, I'm assuming, is first avoid the failure happening at all in the first place. But if it does happen, have it fail in a graceful way that safety is not put at risk. Is that a fair way to describe it? Yeah. That's well, that's the way it used to be until recently. And so classical systems, if you de-energize the system, it's safe. You you have a chemical process and if you can get that thing wound down so it's at room temperature instead of superheated steam, it's a lot safer.