In my last post, I used panel data to conclude that whether you evaluate Donald Trump’s intra-party approval among 2011 Republicans or among 2016/17 Republicans, it remains the same–roughly four out of five Republicans approve of their president. As a result, I rule out any serious concerns with endogeneity of partisanship to approval, which would emerge if original partisans disapprove of Trump at much higher rates. A caveat to the analysis I mentioned is one common to all panel survey analysis: the possibility of nonrandom attrition from the panel. An example could take the form of Republicans who dislike Trump dropping out of the panel (i.e., participating in earlier waves of the panel but not responding to later waves) at a higher rate than their panelist counterparts. That dynamic is certainly plausible. Taking a survey is a political act and means taking time to express one’s opinions on current day politics. Republicans who dislike Trump likely feel some discomfort with current politics and some dissonance–disliking a president from their own party–and thus might avoid expressing themselves politically (i.e., taking a political survey). That distaste with politics may have heightened during the course of 2017 and the several controversies surrounding Trump throughout the year, perhaps after being initially comfortable with discussing their politics (taking a survey) back in 2016.
The same Voter Study Group panel data I used before provides an opportunity to test this idea. While the December 2011 and 2016 waves of the panel survey each includes the same 8,000 Americans who responded to the survey, the July 2017 wave only contains 5,000 of those 8,000 respondents. 3,000 people thus dropped out of the survey. If Republicans who dislike Trump were more likely to leave the panel going from 2016 to 2017 than Republicans who liked Trump, then 2017 intra-party Trump approval might be artificially high. More broadly, evidence of this dynamic could offer insight into another aspect of panel survey attrition–perhaps those experiencing cognitive dissonance as it relates to contemporary politics are especially prone to dropping out.
Focusing on Republicans in the 2016 wave (N = 3,144), I use OLS and logistic regression models to predict a binary outcome: whether an individual took the 2017 survey (a value of 0) or “dropped out” from 2016 to 2017 (a value of 1). (Note that they could still “return” and take future surveys as they did not actually leave the panel itself, I am just using the term “dropout” for shorthand here). Thus, I am predicting panel dropout (relative to continued participation) as the dependent variable. My predictor of interest is “Trump dislike,” captured by a four-point Trump favorability rating (reverse coded so higher values correspond to a more unfavorable opinion) asked in the December 2016 wave of the panel. Motivated by past work on survey panel attrition, I include several control variables in the modeling: gender, education (high school or less, some college, B.A. plus), race (white, black, Hispanic, other), age, four-point political interest scale (higher is more interest), and a partisanship stability variable. I use reported seven-point partisanship from 2011 and 2016 on each individual, and take the absolute value of the difference for this stability variable. Here’s the formula:
- stability = |partyID2011 – partyID2016|
This variable ranges from a value of 0 (perfectly stable party identification from 2011 to 2016) to a value of 6 (switching from Strong Democrat to Strong Republican or vice versa), with a mean of 0.61. Those with the most stable partisan identities should be expected to stay in the panel at highest rates; if this happens to correlate with Trump dislike, it’s further important to include in the modeling.
Below I regress the panel dropout indicator on Trump dislike (Model 1) and then add in the set of control variables (Model 2). Positive coefficients indicate greater likelihood to dropout from the 2016 wave to the 2017 wave.
The statistically significant and positive effect of the Trump dislike variable offers evidence in favor of the nonrandom attrition story motivating this analysis: Republicans less favorable toward Trump were more likely to drop out of the panel. The effect is not overwhelmingly large, and does get cut in half after controlling for other variables, but still remains there. This is important as it shows that the 2017 wave is excluding some Republicans that dislike Trump; if they remained in the panel, Trump’s approval among Republicans would likely be lower in 2017 than what’s actually observed. Attrition thus makes Trump’s Republican approval appear stronger than is really the case (though again, the relationship is not so strong so as to substantially inflate his approval).
To better visualize this main result, I plot the predicted probabilities that a Republican VSG panelist drops out of the survey from 2016 to 2017 as a function of Trump dislike (now using logistic regression instead of a LPM). Control variables are all held at their means or modes.
The probability that a Republican with a very favorable view of Trump drops out of the survey is 0.37, while the probability that a Republican with a very unfavorable view of him drops out is 0.45. Again, while not a substantial increase going from most to least favorable toward Trump, the tendency remains clears–Republicans more unfavorable of Trump dropped out of the panel at a higher rate. Perhaps this results from Republicans who dislike Trump feeling uncomfortable with and avoiding political self-expression, such as by taking a political survey. If that’s the case, this could also be affecting Trump approval surveys more broadly, though that’s more speculative. At the very least, the implications for my earlier analysis are clear: Trump’s approval rating numbers are generally reliable (not suffering from a serious endogenous partisanship problem), but they may still be a little artificially inflated because of this panel attrition problem where Republicans who dislike Trump have become somewhat less willing to take surveys during his presidency.
Agadjanian Politics 