ABC News' Polling Methodology and Standards

Research has found no significant attitudinal biases as a result of response rate differences. A study published in 2000, “Consequences of Reducing Nonresponse in a National Telephone Survey” (Keeter, Miller, Kohut, Groves & Presser, POQ 64:125-48), found similar results in surveys with 61 and 36 percent response rates. A follow-up in 2006, “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey” (Keeter, Kennedy, Dimock, Best & Craighill, POQ 70:759-79), based on surveys with 50 and 25 percent response rates, again found “little to suggest that unit nonresponse within the range of response rates obtained seriously threatens the quality of survey estimates.” Still another Pew comparison, in 2012, with a yet lower response rate, had similar results. As far back as 1981, in “Questions & Answers in Attitude Surveys,” Schuman and Presser, describing two samples with different response rates but similar results, reported (p. 332), “Apparently the answers and associations we investigate are largely unrelated to factors affecting these response rate differences.”

Among many other sources, in "The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms,” in Advances in Telephone Survey Methodology, Chapter 23, Wiley 2007), Holbrook, Krosnick and Pfent reported that “lower response rates seem not to substantially decrease demographic representativeness within the range we examined. This evidence challenges the assumptions that response rates are a key indicator of survey quality.”

Pre-election Polls

Pre-election polling presents particular challenges. As Election Day approaches these polls are most relevant and accurate if conducted among voters. Yet actual voters are an unknown population – one that exists only on (or, with absentees, shortly before) Election Day. Pre-election polls make their best estimate of this population.

Our practice for ABC News is to develop a range of “likely voter” models, employing elements such as self-reported voter registration, intention to vote, attention to the race, past voting, age, respondents’ knowledge of their polling places and political party identification. We evaluate the level of voter turnout produced by these models and diagnose differences across models when they occur.

The use of political party identification in likely voter models is a subject of debate among opinion researchers. It’s used commonly by campaign pollsters, less so among academic researchers. After extensive evaluation we have employed party ID as a factor in some likely voter models for our general election tracking polls, chiefly to adjust for trendless night-to-night variability in political partisanship. (A tracking poll is a series of consecutive, one-night standalone polls reported in a multi-night rolling average.)

ABC News has presented detailed evaluations of our tracking polls at polling conferences and in published work (Langer and Merkle 2001; Merkle, Langer and Lambert 2005; also in Public Opinion Polling in a Globalized World, Springer 2008; Langer et al. 2009).

Join the Discussion
blog comments powered by Disqus
You Might Also Like...