It has become somewhat of a theme of our live blogs this primary calendar to note that Trump's challengers are beating their polls. This first looked to be the case in New Hampshire, where polls significantly underestimate support among Republican likely voters for Haley. But the pattern is more robust than one state: In fact Haley has been beating her polls by an average of five points (in vote share) once you account for undecided voters in these surveys. The consistency of this pattern is striking enough to warrant a post on the matter.
As Mary said earlier, polling primaries is hard! That is betrayed by the fact that POTUS primary polls are by far the least accurate type of poll in 538's pollster rating database going back to 1999. But error is different than bias, and when most of the polls are off in the same direction, something has gone awry.
That "something" is likely the difficulty in obtaining opinions of moderate Republicans from samples of "likely Republican primary voters." Remember that fewer than one percent of people called for a poll actually complete the interview. That means the ones that do are statistical "weirdos" (excuse the technical language). Pollsters adjust for this by weighting their samples to known population benchmarks — like the percent of all adults who are white, over 65, have a college education etc. But in primaries, such benchmarks do not actually exist; pollsters are just making educated guesses about them.
My theory is that most of these primary polls pulling samples of voters from voter registration lists are missing moderate crossover partisans and first-time voters. Additionally, we know that people who are highly motivated to participate in polls (the "weirdos") also happen to be the most politically and ideologically extreme Americans. That's a recipe for polling bias in primaries, where weighting to party, past vote and polarized demographic benchmarks does not control for the partisan consequences of overrepresenting politically engaged Americans.
Now, none of this is to say that polling is "broken." It's just hard to precisely sample a population that doesn't really exist. The polls for the 2024 primary still have below-average error, historically speaking, so we shouldn't go throwing out the baby with the bathwater. But this could nevertheless be a sign of pollsters having a hard time reaching moderate "normie" voters. And if that persists, it could have consequences for general election polls too.
—G. Elliott Morris, 538
