Who's Counting: Which 'Experts' Make Better Political Predictions?

Nov. 5, 2006 — -- An election is coming up, and pundits, commentators and assorted pooh-bahs are full of predictions.

"Expert Political Judgment," a recent book by University of California psychologist Philip Tetlock, suggests that we shouldn't be too impressed by these predictions. With some exceptions, they're not much better than those generated by a good pair of dice.

Tetlock convinced almost 300 experts to make quantifiable predictions about events that might or might not come to pass. The people, all of them professionals paid for "commenting or offering advice on political and economic trends," made more than 80,000 predictions over the period of the study.

Some of their predictions are only slightly disguised versions of empty forecasts, like "Things to Stay the Same Until They Change" or "Things to Change Eventually."

Many of the experts' mistakes result from common psychological foibles that afflict most people. One is the so-called Linda problem. A woman, Linda, is described as very bright, single, outspoken, a philosophy major and an anti-nuclear activist. Given this background, you are asked to say which of the following descriptions of her is more likely: (a) she's a bank teller, or (b) she's a bank teller who is an active feminist.

Most people will choose "b," because the feminist characterization fits comfortably with her background. Nevertheless, "a" is the more likely description, because for it to be true, only the condition that she's a bank teller needs to be met. In order for "b" to be true, however, two conditions must be met -- that she's a bank teller and that she's an active feminist. Satisfying two conditions is always less likely than satisfying one.