The controversy surrounding Roy Moore in the Alabama Senate election seemed primed to create a differential partisan nonresponse bias phenomenon often seen in past election polling. As negative news increases about a candidate in an election, members of the public who share that candidate’s party or support the candidate become less inclined to take polls about the election. Shifts in election polling could thus result from mirages related to these nonresponse patterns, and not actual changes in opinion.
I’ve shown a similar trend using crosstabs from public polling in the context of Donald Trump approval rating polls, finding a moderately strong relationship between the partisan composition of a sample and Trump approval. I thought it might be informative to do the same with the string of pre-election polling for the upcoming Alabama Senate election. First, among polls that make partisan composition data available, I graph the relationship between partisan composition (the difference in Republican and Democratic percentage of a poll’s sample) and Moore’s margin of support (the difference in Moore’s and Jones’s intended vote shares).
There’s a weak but present association between the two variables here. If the relationship was one-to-one, then partisan composition would completely shape the polling outcome and would thus suggest changes in Moore’s polling numbers are shaped entirely to how many Democrats and Republicans take a poll. That’s not the case here For every one point increase in net Republican margin, there’s a 0.36 point increase in Moore’s margin. It’s positive as expected–the more Republicans that take a poll, the better Moore fares–but not too strong. This may have more to do with other aspects by which polls conducted for this race differ, in which case a comparison using consecutive survey results from the same pollster would be a more accurate test. However, not enough polls exist to do that.
Comparing the level of Trump support in a poll with Roy Moore’s advantage over Doug Jones produces a slightly stronger relationship, as shown below:
In this case, for every one point increase in Trump’s net approval, a 0.89 point increase in Moore’s margin over Jones results. Thus, the more people with favorable views toward Trump respond to Alabama Senate election polls, the better Moore appears positioned in the race. It’s worth noting that the variation in Trump net approval across polls–as small as +5 and as large as +22–likely also has to do with how different pollsters approach sampling and vary in methods. If I try to hold pollster “constant,” I only have two JMC Analytics polls to consider. As I’ve mentioned before, the lack of Trump approval change but presence of a Moore margin shift suggested the vote choice opinion shift was not artifactual but real opinion change.
Ideally, there would be more opportunities to look at within-pollster change like this. One comparison doesn’t preclude the possibility of nonresponse bias affecting polling in this race, but should indicate this bias isn’t as clear-cut and strong–even though the race’s dynamics and surrounding news would make it likely. At the same time, looking across polls does reveal patterns indicative of some stereotypical nonresponse bias effects. Polling will likely remain limited down the stretch of this race, but more polls to examine will always give a clearer picture of whether this bias is at play–especially from pollsters who have already polled the race earlier.
Agadjanian Politics 