Exit Poll Response Rates: Don’t Bring Me Down

Aug 28, 2009 10:53am

Response rates to the 2008 national exit poll fell by 9 points compared with 2004, with larger, double-digit declines in 17 state exit polls, according to data from Edison Research, which conducts the exit polls for a media consortium including ABC News.

While lower response rates may not necessarily affect data quality, as high a response rate as practicable is considered desirable in polling, and the drop-offs underscore some of the challenges in conducting the exit polls. Response rates fell to 40 percent or lower in eight states; just one, Indiana, went that low in 2004.

Edison reported a 44 percent response rate to the national exit poll, down from 53 percent in 2004. In all exit poll precincts – including those sampled for individual state exit polls – it reported a 46 percent response rate, also down from 53 percent four years earlier. See the table here.

There was no significant drop-off in exit poll response rates from 2000 to 2004, but there was a decline in 2006. According to the Edison data, the 2008 response rates were much like those in 2006, raising hopes the decline may have been halted. However, the absence of a presidential contest in 2006 makes its comparability unclear.

There are two aspects of exit poll response rates – the refusal rate, meaning the proportion of people who are invited to take the exit poll but decline; and the “miss rate,” the proportion who should have been randomly selected to participate but were missed by the interviewers. Both rose: Edison reported an overall increase from 2004, across all precincts, of 4 percentage points in the refusal rate and 3 points in the miss rate. (Miss rates improved in five states where court challenges successfully overturned restrictions keeping exit poll workers more than 100 feet from polling places.)

Some of the state-level declines in overall response rates were dramatic – a 23-point drop in Nebraska, where the exit poll response rate went from 67 percent in 2004 to 44 percent in 2008; a 20-point drop in Kansas; 18 points in Montana; 17 points each in Alabama, Arizona and Virginia and 16 points in Idaho and Massachusetts.

These response rates refer only to the in-person interviews conducted at polling places. Exit polls in 15 states with high levels of absentee or early voting were supplemented with telephone surveys. (In three other states the “exit polls” were done entirely by phone.) Edison has not yet reported the response rate in its telephone surveys, nor the overall response rate in the exit polls once in-person and telephone samples are combined.

Vote estimates from exit polls often are not accurate until they’re adjusted to actual vote totals. In 2004, for instance, the national exit poll, before weighting to actual vote, overstated John Kerry’s support by 3 points and understated George W. Bush’s support by 3 points. The network consortium that sponsors the exit poll, the National Election Pool, has not released comparable data, based on what’s known as “within-precinct error,” for 2008. The NEP is comprised of ABC News, CBS News, CNN, Fox News, NBC News and The Associated Press.

The role of response rates in exit poll accuracy is not clear; the challenge is not so much the level of nonresponse but differential nonresponse, the propensity of specific groups to be more or less likely to participate. A report on the 1994, 1996 and 1998 exit polls, and a subsequent evaluation of the 2004 exit polls, found no relationship between their response rates and error in their vote estimates.

You are using an outdated version of Internet Explorer. Please click here to upgrade your browser in order to comment.
blog comments powered by Disqus