Sampling Error: What it Means

Note that the division is premised on simple dichotomous responses (support/oppose, yes/no, Candidate A/Candidate B). The formula is different for measures that have three or more response choices – relevant, for instance, in calculating the margin of error for candidate support in a multi-candidate election. While the differences usually are minor for responses in the 30 percent to 70 percent range, for precision in such cases we use a formula reported by Prof. Charles Franklin of the University of Wisconsin in his 2007 paper, "The Margin of Error for Differences in Polls."

The calculation of differences between two independent samples – such as change from one poll to the next – also is computed differently. For example, it takes a change of 4.5 points from one poll of 1,000 to another the same size to be statistically significant, assuming 50/50 divisions in both samples and a 95 percent confidence level. (Again that is lower at different divisions in opinion and/or lower confidence levels; and higher for smaller sample sizes, e.g. subgroups.)

Other comparisons require other calculations. To compare results measured on the difference from one poll to another – e.g., from a 14-point lead for Candidate A in one survey to a 4-point lead for Candidate B in the next – our approach is to calculate the error margin for change in Candidate A's support from one poll to the next, then the error margin for change in Candidate B's support, and ensure that the change is significant.

Calculating the significance of poll-to-poll change in an index, such as the ongoing ABC News Consumer Comfort Index, also requires more complicated calculations, for which ABC relies on consultations with sampling statisticians.

In all cases, the ABC News Polling Unit describes differences or changes in polling data as statistically significant only on the basis of calculations that this is the case. Results that are significant at a high level of confidence, but below 95 percent, may be characterized with modifying language, such as a "slight" change. And in some cases we'll report the confidence level at which a result is statistically significant.

It should be noted that results are not equally likely to fall anywhere within a margin of sampling error, but instead are least likely to extend to its extremes. For example, if candidate support is 51-45 percent in a 772-voter sample with a 3.5-point error margin, that's "within sampling error;" it could be a 46.5-49.5 percent race at the extremes. However, the probability that the result in fact constitutes a lead for the 50-percent candidate can be calculated; in this example it's 91 percent.

That or any confidence level indicates the number of times a theoretical infinite number of samples, of a given size and a given result, would come within sampling error of the actual population value – 9 times out of 10 at 90 percent confidence, 19 out of 20 at 95 percent confidence, 99 out of 100 at 99 percent confidence.

Page
  • 1
  • |
  • 2
  • |
  • 3
Join the Discussion
You are using an outdated version of Internet Explorer. Please click here to upgrade your browser in order to comment.
blog comments powered by Disqus
 
You Might Also Like...