Republican Trump Support and Survey Panel Attrition

In my last post, I used panel data to conclude that whether you evaluate Donald Trump’s intra-party approval among 2011 Republicans or among 2016/17 Republicans, it remains the same–roughly four out of five Republicans approve of their president. As a result, I rule out any serious concerns with endogeneity of partisanship to approval, which would emerge if original partisans disapprove of Trump at much higher rates. A caveat to the analysis I mentioned is one common to all panel survey analysis: the possibility of nonrandom attrition from the panel. An example could take the form of Republicans who dislike Trump dropping out of the panel (i.e., participating in earlier waves of the panel but not responding to later waves) at a higher rate than their panelist counterparts. That dynamic is certainly plausible. Taking a survey is a political act and means taking time to express one’s opinions on current day politics. Republicans who dislike Trump likely feel some discomfort with current politics and some dissonance–disliking a president from their own party–and thus might avoid expressing themselves politically (i.e., taking a political survey). That distaste with politics may have heightened during the course of 2017 and the several controversies surrounding Trump throughout the year, perhaps after being initially comfortable with discussing their politics (taking a survey) back in 2016.

The same Voter Study Group panel data I used before provides an opportunity to test this idea. While the December 2011 and 2016 waves of the panel survey each includes the same 8,000 Americans who responded to the survey, the July 2017 wave only contains 5,000 of those 8,000 respondents. 3,000 people thus dropped out of the survey. If Republicans who dislike Trump were more likely to leave the panel going from 2016 to 2017 than Republicans who liked Trump, then 2017 intra-party Trump approval might be artificially high. More broadly, evidence of this dynamic could offer insight into another aspect of panel survey attrition–perhaps those experiencing cognitive dissonance as it relates to contemporary politics are especially prone to dropping out.

Focusing on Republicans in the 2016 wave (N = 3,144),  I use OLS and logistic regression models to predict a binary outcome: whether an individual took the 2017 survey (a value of 0) or “dropped out” from 2016 to 2017 (a value of 1). (Note that they could still “return” and take future surveys as they did not actually leave the panel itself, I am just using the term “dropout” for shorthand here). Thus, I am predicting panel dropout (relative to continued participation) as the dependent variable. My predictor of interest is “Trump dislike,” captured by a four-point Trump favorability rating (reverse coded so higher values correspond to a more unfavorable opinion) asked in the December 2016 wave of the panel. Motivated by past work on survey panel attrition, I include several control variables in the modeling: gender, education (high school or less, some college, B.A. plus), race (white, black, Hispanic, other), age, four-point political interest scale (higher is more interest), and a partisanship stability variable. I use reported seven-point partisanship from 2011 and 2016 on each individual, and take the absolute value of the difference for this stability variable. Here’s the formula:

• stability = |partyID₂₀₁₁ – partyID₂₀₁₆|

This variable ranges from a value of 0 (perfectly stable party identification from 2011 to 2016) to a value of 6 (switching from Strong Democrat to Strong Republican or vice versa), with a mean of 0.61. Those with the most stable partisan identities should be expected to stay in the panel at highest rates; if this happens to correlate with Trump dislike, it’s further important to include in the modeling.

Below I regress the panel dropout indicator on Trump dislike (Model 1) and then add in the set of control variables (Model 2). Positive coefficients indicate greater likelihood to dropout from the 2016 wave to the 2017 wave.

The statistically significant and positive effect of the Trump dislike variable offers evidence in favor of the nonrandom attrition story motivating this analysis: Republicans less favorable toward Trump were more likely to drop out of the panel. The effect is not overwhelmingly large, and does get cut in half after controlling for other variables, but still remains there. This is important as it shows that the 2017 wave is excluding some Republicans that dislike Trump; if they remained in the panel, Trump’s approval among Republicans would likely be lower in 2017 than what’s actually observed. Attrition thus makes Trump’s Republican approval appear stronger than is really the case (though again, the relationship is not so strong so as to substantially inflate his approval).

To better visualize this main result, I plot the predicted probabilities that a Republican VSG panelist drops out of the survey from 2016 to 2017 as a function of Trump dislike (now using logistic regression instead of a LPM). Control variables are all held at their means or modes.

The probability that a Republican with a very favorable view of Trump drops out of the survey is 0.37, while the probability that a Republican with a very unfavorable view of him drops out is 0.45. Again, while not a substantial increase going from most to least favorable toward Trump, the tendency remains clears–Republicans more unfavorable of Trump dropped out of the panel at a higher rate. Perhaps this results from Republicans who dislike Trump feeling uncomfortable with and avoiding political self-expression, such as by taking a political survey. If that’s the case, this could also be affecting Trump approval surveys more broadly, though that’s more speculative. At the very least, the implications for my earlier analysis are clear: Trump’s approval rating numbers are generally reliable (not suffering from a serious endogenous partisanship problem), but they may still be a little artificially inflated because of this panel attrition problem where Republicans who dislike Trump have become somewhat less willing to take surveys during his presidency.

Does Endogenous Partisanship Distort Trump Approval Numbers among Republicans?

Donald Trump often receives high approval marks from members within his own party, a sign many interpret as a forceful demonstration of strong party loyalty in the current age. Moreover, many view this strong base support as a constraint on other Republican elites; despite a tumultuous presidency, elected Republican officials must heed the opinion of their rank-and-file and cannot abandon Trump. However, many have also raised questions over the reliability of these intra-party approval numbers. Specifically, a key question–that I have tried to speak to before–is whether partisanship is endogenous to Trump approval. If original Republicans who approve of Trump continue to identify as Republican but original ones who disapprove of Trump start to eschew this label, then this creates a misleading portrait of base support. In such a case, Republican identification starts to become inseparable from support for Trump, and thus party breakdowns of Trump approval lose meaning.

Recent panel survey data from Democracy Fund’s Voter Study Group can shed light on this important question. I’ve used this data often, but crucially, this recent release includes a wave in July of 2017. It of course retains its panel structure, providing measures of partisanship in December of 2011, December of 2016, and July 2017 on the same 5,000 individuals. Below, I break down Trump approval (approve and disapprove percentages) asked in the July 2017 wave by these three measures of partisanship. This allows for an intra-party measure of Trump support that few if any current polls can manage–specifically, answering the question of how differently Republican Trump approval (during his presidency) would look like among individuals who originally identified as Republican (i.e., an earlier measure of partisanship, such as in 2011).

The key comparison is between 2017 Trump approval among “original Republicans” (individuals who identified as Republicans in 2011) and 2017 Trump approval among “current Republicans” (those a part of the party in 2017). The more favorable current Republicans’ approval is than original Republicans’ approval, the more evidence accrues in favor of the theory of endogenous partisanship inflating Trump’s base support. If approval ratings among the two groups are similar, then perhaps endogenous partisanship does not present much of a concern. Approval is indeed worse when using this original party ID measurement (79 percent) than when using contemporaneous partisanship to break down Trump approval (83 percent). These percentages are statistically significantly different at p < .01.

However, in a substantive sense, these approval levels are very similar. About four of five Republicans approve of Trump regardless of whether current or earlier measures of people’s partisanship are used. This suggests that partisanship for the most part is not that endogenous to Trump approval, and that high base support for Trump observed in current approval polls during his presidency is not inflated (e.g., HuffPost Pollster data on Republican approval of Trump currently pegs him at 83 percent approve/15 percent disapprove). Caveats–this is just one survey, the most recent Trump approval measure is from July 2017 and things may have changed since then, and possible survey attrition interfering with results–are always in order, but political observers should largely view Trump base support numbers as meaningful.

Charting Recent Racial Attitude Change among Democrats

Even before racial attitudes became particularly important in shaping vote choice in the 2016 election (Sides 2017; Tesler 2016a, 2016b), opinion cleavages between the two parties were forming on issues of race. Data from Pew Research, for example, shows an increase from 28 to 64 percent of Democrats saying “racial discrimination is the main reason why black people can’t get ahead these days” from 2010 to 2017. The shift among Republicans proved much more muted over this seven-year span, with an increase of nine to 14 percent. Not only has this pattern introduced another key area of division between mass public partisans, but it also shows a strong trend toward racial liberalism among Democrats perhaps ushered in by ongoing racial justice causes, such as the Black Lives Matter movement.

Other cross-sectional data sources, such as the American National Election Study, shows a similar pattern–an especially striking recent shift among Democrats on issues of race:

An important question emerges from trends like these. Because of the cross-sectional nature of these data sources, it is unknown if Democrats are changing their minds on racial issue, or if people are keeping the same opinion but changing their partisanship to accord with their preexisting opinion. In the latter case, which would indicate a party sorting dynamic, we would not observe the Pew or ANES time series trends as a result of individuals changing their opinion. Rather, we would explain it as racially conservative 2010 Democrats gradually leaving the party (and thus their racially conservative opinion on this issue no longer gets classified among Democrats) while racially liberal individuals who did not identify as Democrats in 2010 gradually entering the party (and thus making the overall party opinion more liberal on this racial issue).

To understand what dynamic has occurred to a greater degree to result in a Pew-style time series trend (i.e. Democrats growing more racially liberal), I turned to the Voter Study Group panel data. This dataset has key features–namely, it tracks the same 8,000 individuals over a five-year time span, recording attitudinal measures on them in 2011, 2012, and 2016. Crucially, these waves include measures of partisanship and a racial resentment four-question battery. (While often used to capture racial attitudes in political science research, it’s worth noting it has come under criticism for not capturing anti-black racial sentiment so much as general conservatism and individualism. However,  the battery come closest to the type used in the Pew data, and in that sense, using these questions fulfills the goal of my analysis here.)

Here are the four battery items (survey-takers responds with the degree to which they agree with the statements):

1. Over the past few years, Blacks have gotten less than they deserve.
2. Irish, Italian, Jewish, and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.
3. Irish, Italian, Jewish, and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.
4. It’s really a matter of some people not trying hard enough; if Blacks would only try harder they could be just as well off as whites.

Before exploiting the panel structure, I test whether trends in the VSG data on these questions mirror the Pew trend (which is based on a similar but not identical question). Thus, I start with a a cross-sectional format: 2011 racial attitude opinion broken down by 2011 partisanship, and 2016 racial attitude opinion broken down by 2016 partisanship. The VSG data, shown in the below graph, tells a similar story in a growth towards more racially liberal views. For each battery item, I compute the net agreement level (percentage agreeing with the statement minus percentage disagreeing with it), among Democrats only (in blue) and all Americans (in orange).

For each question, people move more liberal on race, but Democrats due so at a faster pace (as expressed by the steeper lines). For example, in 2011, a net +15 Democrats agreed that generations of slavery and discrimination have made it difficult for blacks (i.e. 15 percent more Democrats agreed than disagreed). In 2016, that became a net +38. Among all Americans, the growth was -17 to -12 from 2011 to 2016. While five points in the liberal direction, that still lagged behind the 23 point shift leftward among Democrats on this issue of race. Similarly, on the question of whether blacks have gotten less than they deserve, Americans as a whole got 23 points more liberal while only Democrats got 43 points more liberal from 2011 to 2016. Democrats grew 24 points more liberal on whether blacks should work their way up without special favors (i.e. disagreeing with that statement to a greater extent), while all Americans grew just 10 points more liberal. Finally, on whether blacks could be just as well off as whites if they tried harder, Democrats became 13 points more liberal while all Americans became just one point more liberal.

All told, there have been considerable shifts leftward on racial questions, with Democrats growing liberal in these areas at a faster pace than the general population. Given this, I now move on to the main question: how much individual level change over time is occurring? In order to capture this, I first want to hold partisanship fixed and examine racial attitudes based on the expressed attitudes five years earlier. In that vein of thought, for individuals who identified as Democrats in both 2011 and 2016 (“consistent Democrats”), I plot the percentage agreeing with each racial resentment battery item in 2016 based on their response to the same battery item in 2011.

The results in the graph show greater shifts in the liberal direction on these racial questions in a way that provides evidence for the individual level change story. I’ll walk through the first item–which asks respondents if they agree that blacks have gotten less than they deserve over the past few years–as an example. Here, agreement constitutes the more liberal response, signalling compassion toward blacks and awareness of injustices directed toward them. Among Democrats who agreed with the statement in 2011 (took the liberal position), 86 percent agreed in 2016 (took the liberal position again five years later). However, among those who originally expressed a conservative position–saying in 2011 that they disagreed that blacks have gotten less than they deserved–there was greater decay: only 52 percent of original conservative position-takers held the same conservative position in 2016, while 37 percent flipped to a liberal position (saying they agreed with the statement). With just eight percent of original racial liberals on this issue switching to the conservative side, it’s clear that the individual level change is disproportionately in the liberal direction.

Very similar patterns materialize for the three other battery items. On whether blacks should work their way up without special favors, 88 percent of Democrats taking a liberal position stay liberal five years later while fewer Democrats taking a conservative position at 70 percent do so. Meanwhile, 23 percent of original racial conservatives on this issue become racially liberal on this issue–greater than the eight percent of 2011 racially liberals who become racially conservative on this issue in 2016. These disproportionate “defection rates”–conservative position-takers switching to liberal position-taking and vice versa–appear for the final two items as well. On the issue of whether slavery and discrimination have made it difficult for blacks to work their way out of the lower class, more original conservatives switch to a liberal position (33 percent) than original liberals switching to a conservative stance (nine percent). And on the issue of whether blacks could be just as well of as whites if they tried harder, 25 percent of original racial conservatives became liberal while only eight percent of original racial liberals became conservative–again, confirming the disproportionately greater trend in the liberal direction on racial issues among Democrats from 2011 to 2016. Thus, although most Democrats retain the same racial attitude, the fact that these defection rates are dissimilar (and don’t just cancel out, for example) suggests individual level “changing of minds” is contributing to patterns of aggregate racial attitude change among Democrats.

This of course doesn’t rule out other explanations for the Pew time series trend, as I discuss above. Perhaps partisans have been sorting around anti-black resentment levels in the last five years, with racial conservatives keeping their opinion but shifting to the Republican Party and racial liberals maintaining the same stance but entering the Democratic Party. While a plausible theory given the general importance of racial attitudes as well as past similar phenomena such as white southern partisan realignment, this sorting hypothesis does not receive much support from the VSG panel data. To test this idea, I now want to hold racial attitudes constant while checking partisanship in 2016 based on one’s expressed partisanship in 2011. I thus ask how does partisanship among consistent racial conservatives (individuals taking conservative positions on an item in both 2011 and 2016) and consistent racial liberals (individuals taking liberal positions on an item in both 2011 and 2016) look like in 2016 vs. 2011. Are racial conservatives in the Democratic Party leaving the party at disproportionate rates? Similarly, are racial liberals in the Republican Party leaving the party at disproportionate rates? The two below graphs–the first holding racial conservatism constant and the second holding racial liberalism constant–address these questions.

From a qualitative standpoint, there does appear to be less evidence of party sorting around the racial resentment battery items than there is individual opinion change expressed in the earlier graph. However, sorting does occur to some extent, as in every case there is some (just not large) partisanship change consistent with this hypothesis. I’ll use the battery item about whether blacks should overcome prejudice and work their way up without special favors as an example. Among consistent racial conservatives, more 2011 Republicans retained the same partisan identity in 2016 (91 percent) than 2011 Democrats did in 2016 (79 percent). On the other hand, among consistent racial liberals on this issue (saying they disagreed with the statement), 97 percent of Democrats retained the same partisanship while fewer Republicans at 80 percent did so. These slightly disproportionate “defection rates” make sense in light of the sorting hypothesis–racial conservatives have increasingly left the Democratic Party and racial liberals have increasingly left the Republican Party.

Other battery items reveal similar magnitude levels. But while partisanship change observed here confirms some presence of sorting around racial resentment levels, the magnitude of change appears below that of opinion shift among original Democrats explained earlier, for example. It’s worth noting that comes from more of a qualitative view, however–I’m unaware of quantitative approaches that can directly pinpoint how much of overall opinion change is due to individual level change versus sorting. In sum, in explaining the recent shift leftward on issues of race among Democrats, there’s evidence of both individual level changing of minds and partisan sorting around issues of race within the last decade, but likely more so in favor of the former.

---

For more on who’s driving the Democratic shift toward racial liberalism, see my piece with Sean McElwee as well as supplementary materials here.