ABC News' Polling Methodology and Standards

A summary of ABC News polling standards and methodology follows.

Standards
The ABC News Polling Unit vets all survey research presented to ABC News to ensure it meets our standards for disclosure, validity, reliability and unbiased content. We recommend that our news division not report research that fails to meet these standards.

On disclosure, in addition to the identities of the research sponsor and field work provider, we require a detailed statement of methodology, the full questionnaire and complete marginal data. If any of these are lacking, we recommend against reporting the results. Proprietary research is not exempted.

Methodologically, in all or nearly all cases we require a probability sample, with high levels of coverage of a credible sampling frame. Self-selected or so-called "convenience" samples, including internet, e-mail, "blast fax," call-in, street intercept, and non-probability mail-in samples do not meet our standards for validity and reliability, and we recommend against reporting them.

We do accept some probability-sample surveys that do not meet our own methodological standards – in terms of within-household respondent selection, for example – but may recommend cautious use of such data, with qualifying language. We recommend against reporting others, such as pre-recorded autodialed surveys, even when a random-digit dialed telephone sample is employed.

In terms of content, we examine methodological statements for misleading or false claims, questionnaires for leading or biasing wording or ordering, and analyses and news releases for inaccurate or selective conclusions.

In addition to recommending against reporting surveys that do not meet these standards, we promote and strongly encourage the reporting of good-quality polls that break new ground in opinion research.

Sampling
Field work for most of our U.S. polling is carried out by TNS of Horsham, Pa., using a dual-frame sample design covering both landline telephone and cell phone-only respondents, with samples produced by Survey Sampling Inc. of Shelton, Conn. We tested cell-only sampling in August 2008 and made it a regular part of our sample design in October 2008. We continue to assess developments and data sources in cell phone sampling and to evaluate our approach as best practices in this area develop.

Landline
In the landline component of these surveys, a sample of landline households in the continental United States is selected by SSI via random digit dialing procedures, in which all landline telephone numbers, listed and unlisted, have an equal probability of selection.

SSI starts with a database of all listed telephone numbers, updated on a four- to six-week rolling basis, 25 percent of listings at a time. This database of directory-listed numbers is then used to determine all active blocks – contiguous groups of 100 phone numbers for which more than one residential number is listed. All possible numbers in active blocks are added to the random digit database.

Until 2005, ABC News followed the industry norm of excluding all listed business numbers (compiled from sources such as Yellow Pages directories and the Dunn and Bradstreet Business Data database) from the sample. However, an ABC-led study (Merkle, Langer, Cohen, Piekarski, Benford and Lambert, 2009, Public Opinion Quarterly) found that this "cleaning" process excludes respondents who have home-based business-listed phones and no other lines at home on which they take calls, creating 3 percent noncoverage of eligible households with no offsetting gains in productivity. As a result of this evaluation, we do not exclude listed business numbers from our landline sample, with the exception of those in business-only blocks or exchanges.

Each telephone exchange in the landline sample is assigned to the county where it's most prevalent. In the first stage of selection, the database is sorted by state and county, and the number of telephone numbers to be sampled within each county is determined using systematic sampling procedures from a random start, such that each county is assigned a sample size proportional to its share of possible numbers. In the second stage of selection, telephone numbers are sorted within county by area code, exchange and active block, and using systematic sampling procedures from a random start, individual phone numbers within each county are selected. The sampled phone numbers are pre-dialed via a non-ringing auto-dialer to reduce dialing of non-working numbers.