Watching the Arab Spring, I have been struck by the remarkable confidence of commentators as they predict the future of the region, even while in disagreement about what that future will be. How could they be so confident when surely some of them must be wrong?
This overconfidence is something very familiar to social psychologists (not just because we sometimes display it, but because we study it!). For instance in one study by David Dunning and his colleagues, students were asked to predict the likelihood of events like whether a person would drop a course, live off campus, etc.. When the students were 100% certain something would happen, it happened 85% of the time. If you've seen the movie Princess Bride, you know that Inigo Montoya gets this discrepancy. Vizzini repeatedly seems certain, but then the unexpected happens, to which Vizzini exclaims "Inconceivable!" Montoya's response is "You keep using that word. I do not think it means what you think it means." Ah, certainty. How little we understand you!
If you want to examine the overconfidence of political commentators, I recommend Philip Tetlock's Expert political judgment: How good is it? How can we know?The first chapter is available (free!) here:
This is an extraordinarily lovely first chapter, though parts get technical in ways that might daunt those of you who aren't social scientists. Tetlock has, for decades, tested political experts; ability to predict the future. Do the experts do better than, say, chimps throwing darts? Well, some seem to exceed that standard, but on the whole, Tetlock has found them far too Vizzini-like, declaring "inconceivable" that which nonetheless occurs.
But Tetlock finds that some forecasters do better than others. Who? Well, first let's look at what does not make a difference. Per Tetlock:
Who </em>experts were--professional background, status, and so on--made scarcely an iota of difference to accuracy. Nor did what experts thought--whether they were liberals or conservatives, realists or institutionalists, optimists or pessimists.
What does matter? Again, Tetlock:
How experts thought--their style of reasoning--did matter. The foxes' self-critical, point-counterpoint style of thinking prevented them from building up the sorts of excessive enthusiasm for their predictions that hedgehogs, especially well-informed ones, displayed for theirs.
In speaking of foxes, Tetlock describes a cognitive style in which one is self-critical, seeking points contrary to what the fox thinks, whereas hedgehogs (think the most ideological of our brethren and sistren) think one big idea and tend to apply it to everything, eschewing contradictory data. This will be a theme I will tackle repeatedly here at Psych Geek. Cognitive style matters. Do we ask ourselves why we might be wrong? Do we grapple with contradictory evidence? Do we tolerate ambiguity? Or do we seek the comfort of certainty? I am a big fan of the honest pursuit of information that conflicts with what we believe, as a way of being more Montoya than Vizzini.
If you're interested in human reasoning, and in trying to bring a critical mind to bear on the vapidity that so often marks contemporary political discourse, I think this is your book. I think there's a 70% chance it is, in fact!
In the meantime, trust the foxes for predictions if you value accuracy. This might mean living with uncomfortable uncertainty. I am more inclined to view it as living with a complexity that makes room for hope.
old post: Overconfidence, Foxes, Hedgehogs, and Inigo Montoya