New Hampshire’s Polling Fiasco

Jan 9, 2008 12:05am

There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.

But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened – about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen – a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities – such as their own failings in sampling and likely voter modeling.

There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.) And there was no overstatement of Obama in Iowa polls.

On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side – where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.

A starting point for this analysis will be to look at every significant Democratic subgroup in the New Hampshire pre-election polls, and see how those polls did in estimating the size of those groups and their vote choices. The polls' estimates of turnout overall will be relevant as well.

In the end there may be no smoking gun. Those polls may have been accurate, but done in by a superior get-out-the-vote effort, or by very late deciders whose motivations may or may not ever be known. They may have been inaccurate because of bad modeling, compromised sampling, or simply an overabundance of enthusiasm for Obama on the heels of his Iowa victory that led his would-be supporters to overstate their propensity to turn out. (A function, perhaps, of youth.)

Prof. Jon Krosnick of Stanford University has another argument: That the order of names on the New Hampshire ballot – in which, by random draw, Clinton was toward the top, Obama at the bottom – netted her about 3 percentage points more than she'd have gotten otherwise. That's not enough to explain the gap in some of the polls, which presumably randomized candidate names, but it might hold part of the answer.

The data may tell us; it may not. What's beyond question is that it is incumbent on us – and particularly on the producers of the New Hampshire pre-election polls – to look at the data, and to look closely, and to do it without prejudging.

Wednesday afternoon p.s.: Some folks are suggesting that "late deciders" made the difference – a common explanation for poor estimates. But the exit poll doesn't support the notion. Remove voters who decided on Tuesday and the New Hampshire exit poll result is Clinton +2 exactly her actual margin. (Among those who decided "just today" it was Clinton +3.) Next theory.

You are using an outdated version of Internet Explorer. Please click here to upgrade your browser in order to comment.
blog comments powered by Disqus