A summary of ABC News polling standards and methodology follows.
Langer Research Associates, primary polling provider to ABC News, advises the news division on standards for disclosure, validity, reliability and unbiased content in survey research, and evaluates data when requested to establish whether it meets these standards.
On disclosure, in addition to the identities of the research sponsor and field work provider, we require a detailed statement of methodology, the full questionnaire and complete marginal data. If any of these are lacking, we recommend against reporting the results. Proprietary research is not exempted.
Methodologically, in all or nearly all cases we require a probability-based sample, with high levels of coverage of a credible sampling frame. Non-probability, self-selected or so-called “convenience” samples, including internet opt-in, e-mail, “blast fax,” call-in, street intercept and non-probability mail-in samples do not meet our standards for validity and reliability, and we recommend against reporting them.
We do accept some probability-based surveys that do not meet our own methodological standards – in terms of within-household respondent selection, for example – but may recommend cautious use of such data, with qualifying language. We recommend against reporting others, such as pre-recorded autodialed surveys, even when a random-digit dialed telephone sample is employed.
Langer Research Associates has published briefing papers summarizing recent research on two forms of non-probability sampling – one on opt-in online surveys and another on the use of social media (and related approaches) to estimate public opinion. We’ve also published a summary of challenges in the use of “big data” and contributed commentary on non-probability sampling in the Fall 2013 issue of the Journal of Survey Statistics and Methodology.
In terms of content, we examine methodological statements for misleading or false claims, questionnaires for leading or biasing wording or ordering, and analyses and news releases for inaccurate or selective conclusions.
In addition to recommending against reporting surveys that do not meet these standards, we promote and strongly encourage the reporting of good-quality polls that break new ground in opinion research.
ABC and The Washington Post direct the methodological approach to full-length ABC/Post polls in consultation with our field work provider for these surveys, Abt SRBI of New York, N.Y. (Shorter-length ABC/Post polls are conducted via the SSRS omnibus survey, produced by SSRS of Media, Pa.)
Full-length ABC/Post polls use a dual-frame sample design covering both landline telephone and cell phone-only respondents, with samples produced by Survey Sampling Inc. (SSI) of Shelton, Conn. We tested cell-only sampling in August 2008, incorporated it into our ongoing samples in October 2008 and reported on our approach in a paper at the annual meeting of the American Association for Public Opinion Research in May 2010.
Our approach is to view landline and cell-phone sampling frames as mutually exclusive; the purpose of the cell-only frame is to address the landline frame’s non-coverage of cell-only households, based on the most recent estimates from the federal government’s in-person National Health Interview Survey.