Survey Monkey does a lot of great, innovative work, but the one thing that continues to pester me is how they present certain results. While they offer a wide array of ways to examine their polling data, party breakdowns of results–how Democrats and Republicans responded to a recent poll question, for example–usually use ungrouped party identification scales. As I’ve discussed in detail, this method of not grouping Independents with the party to which they lean creates a poor measure of party affiliation. It implies Democrat-leaning and Republican-leaning Independents do not have any inclinations toward the major parties and are distinct from partisans in terms of their political behavior. I find that notion is misleading and mistaken for the most part, as these Independent leaners behave like regular partisans along several outcomes, such as vote choice, self-reported ideology, and issue positions. This idea is nothing new, as I review in my past blog post. Political science research has long documented this phenomenon, both the existence of these closet partisans and the misleading nature of classifying them as Independents.
Just as evidence for the importance of grouping leaners (closet partisans) persists, so too do the tendencies on the part of professional pollsters to not adequately account for this. An even partisanship distribution–roughly a third of respondents who are Democrats, Independents, and Republicans–should raise the immediate red flag. When you group Independent leaners with their respective parties and thus follow the more informative method, that distribution levels out to about 40-45 percent for the two parties, and 10-20 percent pure, unaffiliated Independent variables–close to what Pew finds here. That red flag was raised for me concerning a recent Survey Monkey poll and discussion of its results. Here’s the snippet of the results release that caught my eye, regarding support for “the Republican health care plan to repeal and replace the Affordable Care Act”:
- “Yet those with less partisan attachment have largely turned against the bill. Americans who initially identify as independent oppose the Republican health care bill by a large margin (40 percent support, 59 percent oppose). Independents who who lean to neither party are against it by an even larger margin (32 percent support, 61 percent oppose).”
Of course, this instance is not nearly as egregious as others. Survey Monkey does in fact break down results with grouped leaners. But a look at this post’s presentation reveals its emphasis: the first mention of party breakdowns uses the deceptive ungrouped leaner method. The one graph in the blog post for this topic uses ungrouped leaners too. As a result, this type of data on Independents essentially includes respondents who are identical to normal partisans, which clearly should not be the intent of these results.
I’m taking a critical tone here, but among the many things that Survey Monkey excels at is transparency. One component of that is release of detailed demographic breakdowns. The inclusion of an ungrouped, three-point party ID and a fuller five-point party ID scale allowed me examine their methodological decisions more closely and what they decide to stress in presentation of their results. When asked the following question:
- Do you support or oppose the Republican healthcare plan to repeal and replace the Affordable Care Act?
Here’s how results break out if Independent leaners are classified as Independents (making for a three-point PID system with ungrouped leaners):
Again, this method–which Survey Monkey emphasized in this blog post/results release and in ones in the past–necessarily classifies partisan-like respondents as Independents. The other partisan breakdown for opinion on this new healthcare plan provided by Survey Monkey uses a five-point party affiliation scale. While not implementing the usual seven-point scale (and the more effective one for capturing more partisanship variation), this breakdown proves much more meaningful. Namely, it separates out the Independent leaners, which allows us to compare them to self-identifying partisans. Here’s how that breakdown looks:
Note that these responses refer to opinion on the new Republican health care plan. This establishes that more support is the more Republican position, and more opposition assumes the more Democratic stance. When we compare support rates across the five different groups, it becomes very clearly that leaners (“Lean R” and “Lean D”) hold quite similar opinions to those that regular partisans (“Rep.” and “Dem.”) do. Nearly the same amount of self-identifying Republicans (84%) support the plan as Republican-leaning Independents (83%) do. The same closeness in opinion holds for Democratic partisans (7%) and Democratic leaners (10%). It’s thus worth noting how seriously misleading it is to categorize these leaners as Independents, as the previous chart–the one with data Survey Monkey emphasizes most–does. The middle group (“Pure Ind.”), pure Independents, expresses fairly distinct opinions from these surrounding four groups, with 33% support.
The same story emerges when we look at the rates of opposition. Republican partisans (14%) are just about indistinguishable in opinion from Republican leaners (16%). The same holds on the other side of the aisle, where Democratic-leaning Independents in fact express slightly more Democratic views (92% oppose) than self-identifying Democrats do (88%), though the difference here may not be too statistically significant.
The broader point stands: Independents who lean toward a party are very close in opinion–and largely distinguishable–from partisans who identify as such when they’re initially asked about their party ties. Moreover, it’s only after separating out these leaners that the behavior and opinion of the leftover Independents becomes meaningful and closer to a reflection of those who truly don’t hold strong partisan ties. Movement within this specific group is often consequential for political outcomes, a notion that would be difficult to recognize without proper party affiliation categorization.
This, of course, is not to say that only party identification data and breakdowns that use grouped leaners should be reported. Though they behave very similarly to partisans, the fact that they eschew partisan labels in the first place is indeed substantively important. However, when presenting information on opinion splits by party to the public in a concise fashion, it behooves everyone involved–in order to produce the most realistic representation of political behavior–to group leaners with the parties to which they lean, and emphasize this data above all else.
Agadjanian Politics 