Mind the Gap

Dec 12, 2007 1:48pm

Three good national polls this week told similar stories about the presidential race – but with some different numbers. We’re sure to see it again, so it’s worth sorting out.

The most notable differences were in the numbers that get the most attention – the horse race. Part of the reason is in the way we evaluate these numbers; another part, in how they’re produced. And a third thought is about the way we tend to (over-) focus on them.

The Republican numbers are easier to evaluate. A CNN poll had a 2-point race between Rudy Giuliani and Mike Huckabee; a New York Times/CBS poll, a 1-point race; the ABC News/Washington Post poll, a 6-point Giuliani lead (a “lead,” that is, at an 88 percent confidence level – see previous item on that subject).

But focusing on the gap between candidates exaggerates small differences in polls, and misses the real aim of the horse race question – to measure each candidate’s level of support, not the space between them. (A poll with Giuliani at 5 percent and Huckabee at 4 percent is not the same as a poll with them at 22 and 21 percent respectively, even though the gap matches.)

Consider instead the candidates’ actual support levels in these polls: Giuliani, 24, 22 and 25 percent, respectively; Huckabee, 22, 21 and 19. All are well within tolerances for their sample sizes (377, 266 and 292) – and indeed pretty similar, especially given their other differences. These polls were done on the same days, but CNN’s was among registered voters, while the CBS/NYT and ABC/Post polls were among “likely voters,” which each polling outfit defines differently. And the number of “undecideds” – in our view a function of polling technique rather than actual indecision – was 4 percent in the ABC/Post poll, 6 percent in CNN’s, but 17 percent in CBS/NYT’s.

Most important, moreover, is their fundamental message – lower support for Giuliani, higher for Huckabee, with a wealth of data in each of these surveys to help us understand why that is. On that central point, these surveys are in accord.

Numbers on the Democratic race are tougher to parse out. Looking at the gaps makes the differences look garish – a 10-point lead for Hillary Clinton over Barack Obama in the CNN poll, 17 points in CBS/NYT’s, 30 points in ABC/Post’s. Better again to look instead at the candidates’ support levels. John Edwards was at 14, 11 and 10 percent respectively – similar. Obama was at 30, 27 and 23 percent – a significant difference between the high and low estimates. And in the biggest difference, Clinton was at 40, 44 and 53 percent respectively.

There again are differences in populations and in levels of undecideds (in this race, 9 percent undecided in the CBS/NYT poll, vs. 3 percent in ABC/Post’s). Sample management (e.g., number of interviews per night) also can, at times, create differences in results. And differences among groups is another possible cause. With an African-American prominent in the race, ABC/Post polls consistently have been oversampling black respondents all year, in order to increase our confidence in this particular estimate (especially because blacks account for a nearly fifth of all likely Democratic primary voters). We find Clinton ahead of Obama by 52-39 percent among African-Americans in our poll. (At one point last summer, when our data differed from a Gallup poll that had Obama much closer, it looked to us like estimates of preferences among black voters was a likely cause. And we liked the higher confidence we got from our oversample.)

The Democratic race was almost precisely the same in this ABC/Post poll as in our two previous national polls, in early November and late September. CNN’s was 40-30-14 percent, compared to a 44-25-14 result in early November – that 5-point rise for Obama reaches significance. The CBS/NYT results (44-27-11) compared with 51-23-13 in a poll CBS did in mid-October; that earlier poll, however, left out the five second-tier candidates as choices, making comparisons a bit dicier (those five get a net of 7 points in the latest CBS/NYT poll, 8 points in ABC/Post data, and 12 in the CNN poll).

What to conclude? One healthy approach would be to cut back on fixation with the horse race and look at the underlying evaluations. These polls all have Clinton ahead, albeit by different margins. What they do best, in exploring the candidates’ strengths and weaknesses, and the issues motivating voters, is to help us see why.

You are using an outdated version of Internet Explorer. Please click here to upgrade your browser in order to comment.
blog comments powered by Disqus