Polling autopsy shows little evidence of shy Trump voters

— -- A task force of research experts says there's "little backing" in 2016 election polling data for the shy Trump hypothesis — the theory that Donald Trump supporters were unwilling to tell pollsters they favored him because they thought it was socially unacceptable to do so.

A 104-page report from a special American Association for Public Opinion Research panel charged with evaluating political polling after the 2016 presidential election found that national polls were "generally correct" but state-level polling in Rust Belt states crucial to Trump's victory "showed some large, problematic errors" and "failed to adequately measure support for Trump."

"The day after the election, there was a palpable mix of surprise and outrage directed towards the polling community, as many felt that the industry had seriously misled the country about who would win," the panel's report reads. It then dives into a detailed analysis of polling in the lead-up to Election Day.

The Trump campaign urged his supporters not to believe political polls, which showed that Democratic nominee Hillary Clinton was likely to win.

To explain why some state polling did not align with the election's outcome in several key states, the panel, composed of public opinion and survey research experts, pointed to a "real change in vote preference during the final week or so of the campaign," as well as the failure of many state polls to include the correct proportion of non-college graduates — a core Trump constituency.

The data showed that the Trump-Clinton margin in national polls was off by an average of 2.2 percentage points — among the most accurate in U.S. polling back to 1936 — but state-level polls were off by an average of 5.1 percentage points, according to the report.

But the panel found little evidence for one major theory after the election, that voters did not declare their support for Trump in polling because they did not believe it was socially acceptable.

If the shy Trump theory were accurate, experts say, they would expect his support to be lower in live-interviewer polls than in robo-dial polls because people would be more hesitant to tell a human being they supported Trump. But the study's results were "inconsistent with expectations of the shy Trump theory."

The report also examined data for a relationship between the proportion of undecideds in a poll and estimates of Trump support — more evidence of a shy Trump factor. But the study found results were "failing to yield evidence supporting the shy Trump hypothesis."

No 'evidence of bias'

The data do "not show evidence of bias" when it comes to the idea of differential nonresponse, a theory that Trump voters in general were less likely to participate in a poll than people who supported other candidates. But the report says its findings "do not rule out the possibility."

The report also says there is evidence, particularly in the upper Midwest, that voters who told pollsters they were undecided or voting for a third party candidate ended up casting their ballots for Trump.

But the committee doesn't point to FBI Director Jim Comey's decision to reveal new emails in the Clinton email scandal as the main reason for Clinton's late skid.

"We would conclude there is, at best, mixed evidence to suggest that the FBI announcement tipped the scales of the race," the report says. "It is reasonable to speculate that Clinton's slide began as early as Oct. 22 or 23."

The report says state-level polling may have gotten by without weighting its results based on education level in 2012 because the split between voters with and without a college degree was not as drastic. But in 2016, when education was a significant variable, that method "completely fell apart."

"Adjusting for overrepresentation of college graduates was critical, but many polls did not do it," says the report, adding that voters with higher education levels are more likely to participate in a poll than voters without a degree. "Many polls, especially at the state level, did not adjust their weights to correct for the overrepresentation of college graduates in their surveys, and the result was overestimation of support for Clinton."

Swing-state polls missed education

The study showed that the majority of national polls adjusted their results for education but that only one-third of polls or fewer in swing states where Trump won weighted their results for education, including just 18 percent of Michigan polls and 27 percent of Wisconsin polls.

Other factors, like the order of names on the ballot and the possibility that changes in turnout from 2012 to 2016 resulted in inaccurate likely voter models, may have contributed to the problem. But the report added that there is less compelling evidence for such reasons and they did not alone account for the error in state polling.

Based on data from a Pew Research Center callback survey, which calls the same people before and after the election to see whether their vote was consistent with their response in the pre-election poll, 11 percent of people told pollsters one thing before the election but cast their ballot differently. The committee called those numbers "quite typical."

But in 2016, inconsistent respondents favored Trump by a 16 percentage point margin, the largest since Pew began the callback survey in 2000.