ABC News' Polling Methodology and Standards

In spring 2003 ABC News and the Washington Post produced sample dispositions for five randomly selected ABC/Post surveys at the request of Prof. Jon Krosnick, then of Ohio State University, for use in a study of response rates. The cooperation rate calculations produced by Krosnick’s team for these five surveys ranged from 43 to 62 percent, averaging 52 percent; response rates ranged from 25 to 32 percent, based on what AAPOR describes as a “very conservative” estimate of the number of business and nonworking numbers in the sample (known as “e”). The range was 31 to 42 percent using a more common estimate of this variable proposed by Keeter et. al. in 2000.

In their study (“The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms,” in Advances in Telephone Survey Methodology, Chapter 23, Wiley 2007), Holbrook, Krosnick and Pfent concluded, “lower response rates seem not to substantially decrease demographic representativeness within the range we examined. This evidence challenges the assumptions that response rates are a key indicator of survey quality.”

Pre-Election Polls

Pre-election polling presents particular challenges. As Election Day approaches these polls are most relevant and accurate if conducted among voters. Yet actual voters are an unknown population – one that exists only on (or, with absentees, shortly before) Election Day. Pre-election polls make their best estimate of this population.

Our practice at ABC News is to develop a range of “likely voter” models, employing elements such as self-reported voter registration, intention to vote, attention to the race, past voting, age, respondents’ knowledge of their polling places and political party identification. We evaluate the level of voter turnout produced by these models and diagnose differences across models when they occur.

The use of political party identification in likely voter models is a subject of debate among opinion researchers. It’s used commonly by campaign pollsters, less so among academic researchers. After extensive evaluation ABC News has employed party ID as a factor in some likely voter models for our general election tracking polls, chiefly to adjust for trendless night-to-night variability in political partisanship. (A tracking poll is a series of consecutive, one-night standalone polls reported in a multi-night rolling average.)

ABC News has presented detailed evaluations of our 2000, 2004 and 2008 tracking polls at polling conferences and in published work (Langer and Merkle 2001; Merkle, Langer  and Lambert 2005; also in “Public Opinion Polling in a Globalized World,” Springer 2008; Langer et al. 2009) showing that party ID factoring had little effect on our estimates of vote preferences.

With thanks to Linda Piekarski of SSI for review and comment.

Join the Discussion
You are using an outdated version of Internet Explorer. Please click here to upgrade your browser in order to comment.
blog comments powered by Disqus
You Might Also Like...