The Courage to Be Wrong with Dan Gardner

Dan Gardner (0:06)

What's not going to change? Questions, good questions.

My belief is that the value of high quality questions will soar. The value of answers is going to decline. The value of high quality questions is going to soar.

Justin Thomson (0:25)

Welcome to The Angle from T. Rowe Price, a podcast for curious investors. And just a reminder that outside of the US and Australia, this podcast is for investment professionals only. Thank you for joining me today. I'm Justin Thomson, head of the T. Rowe Price Investment Institute. Today I'm thrilled to welcome Dan Gardner to The Angle.

Dan is an award-winning author and an expert on decision-making, risk, and how we navigate uncertainty. He's written four books, including Superforecasting, co-authored with academic Philip Tetlock, which is going to be the focus of our pod today.

Superforecasting explores why some people are consistently better at predicting what happens next and how the principles of superforecasting can help us make better decisions in an uncertain world. I forecast this is going to be a super conversation. Dan, welcome.

Dan Gardner (1:20)

Delighted to be here.

Justin Thomson (1:21)

Dan, I'm here to pick your brain not to plug your book, but I think in superforecasting, you have created something really useful, and at the same time is great entertainment. What we call in the UK a ripping yarn.

So tell us a little bit about the background to the book and what you set out to achieve with it.

Dan Gardner (1:46)

Sure.

It's interesting. I was wondering, where do I start the story? I think I start, oddly enough, with Saddam Hussein. It all started with Saddam Hussein. You'll recall that the invasion of Iraq by the United States in 2003 was predicated on the premise that Saddam Hussein was stockpiling weapons of mass destruction in violation of United Nations sanctions. Then they invaded and they discovered they looked high and low and there were no weapons of mass destruction.

But the officials with the Office of the Director of National Intelligence, which is the base of the agency that oversees the entire community, they recognized that this is, you know, we've got to do better than this, folks. And so they started to look into that and say, well, for our forecasting, is our forecasting as good as it can be? Could we make it better? And so one of the very first things that they did in asking that fundamental question was they went to my co-author, Philip Tetlock, who is at the Wharton School, University of Pennsylvania, a very eminent psychologist. And basically, Phil actually pioneered research on expert forecasting in research that he did in the 80s and 90s. And the first thing that Phil told them was, well, first of all, we actually don't even know how good your forecasting is because you've never subjected it to proper testing, which is bad, right? It's kind of ridiculous, in fact, given how important your forecasting is.

Then Phil suggested basically that they create a forecasting tournament, in which you would have basically competing teams using competing whatever competing methods that they chose to try and forecast real world questions, the sort of very difficult questions that the intelligence community has to answer. And then you would have these people make those forecasts. You could then, after time passed, you could judge the accuracy of the forecasting. You could compare what was working, what wasn't working and so on. Long story short, they end up creating five university based research teams. They compete on these real world forecasting questions.

The team that was created by my co-author, Philip Tetlock, was called The Good Judgment Project. And out of that group, they used basically volunteers. They asked for volunteers to make geo-political forecasts. And in that group, there was a very small percentage, you know, one to two percent of these people who were consistently top of the charts. They were really, really impressively good. I can give you all sorts of metrics to show how good they were, but they were just amazingly good. And that's the term, hence the term Superforecaster simply means top of the charts. It sounds a little grandiose when you say Superforecaster, but that's just means top of the charts. And I emphasize that these people were consistently good. Very key, because of course, when you're consistently good over time, you can be confident that you're looking primarily at skill, not luck. But intellectual humility is absolutely foundational to good forecasting judgment. Good forecasters are at root intellectually humble. They know that the world is a big, messy, complicated place, and forecasting is a damned hard business.

Justin Thomson (5:16)

As is the case in our business, Dan, as is the case.