ABC News' Polling Methodology and Standards

A summary of ABC News polling standards and methodology follows.

Standards

Langer Research Associates, primary polling provider to ABC News, advises the news division on standards for disclosure, validity, reliability and unbiased content in survey research and evaluates data when requested to establish whether it meets these standards.

On disclosure, in addition to the identities of the research sponsor and field work provider, we require a detailed statement of methodology, the full questionnaire and complete marginal data. If any of these are lacking, we recommend against reporting the results. Proprietary research is not exempted.

Methodologically, in all or nearly all cases we require a probability-based sample, with high levels of coverage of a credible sampling frame. Non-probability, self-selected or so-called “convenience” samples, including internet opt-in, e-mail, “blast fax,” call-in, street intercept and non-probability mail-in samples do not meet our standards for validity and reliability, and we recommend against reporting them.

We do accept some probability-based surveys that do not meet our own methodological standards – in terms of within-household respondent selection, for example – but may recommend cautious use of such data, with qualifying language. We recommend against reporting others, such as pre-recorded autodialed surveys, even when a random-digit dialed telephone sample is employed.

Langer Research Associates has published briefing papers summarizing recent research on non-probability sampling, including opt-in online surveys, the use of social media (and related approaches) to estimate public opinion and challenges in the use of “big data.” We’ve also commented on non-probability sampling in the Fall 2013 issue of the Journal of Survey Statistics and Methodology and in a 2012 presentation at the annual conference of the American Association for Public Opinion Research.

In terms of survey content, we examine methodological statements for misleading or false claims, questionnaires for leading or biasing wording or ordering, and analyses and news releases for inaccurate or selective conclusions.

In addition to recommending against reporting surveys that do not meet appropriate standards for validity and reliability, we promote and strongly encourage the reporting of good-quality polls that break new ground in opinion research.

Sample Design

ABC and The Washington Post direct the methodological approach to full-length ABC/Post polls in consultation with our field work provider for these surveys, Abt Associates of Rockville, MD. Occasional shorter-length ABC/Post polls are conducted via an omnibus survey produced by SSRS of Glen Mills, PA; see methodological details here.

Before October 2008, full-length ABC/Post polls were conducted by calling samples of landline telephone numbers only. From October 2008 through June 2015 we added cell phone interviews via a non-overlapping dual-frame sample design, with separate sampling frames for landline and cell phone-only respondents, as detailed in this paper. The cell phone-only proportion, based on data from the National Health Interview Survey, grew from 100 out of 1,000 interviews to 335 per 1,000 during this period.