Historical Trends in White Political Behavior Along Educational Lines, and Where 2016 Fits In

A longstanding topic of interest, the voting behavior of working class white population–and socioeconomic divides in voting patterns more broadly–once again attracted considerable attention during and after the 2016 election. Some assessments that have historically contextualized the low SES white vote have showed that this group voted more Republican than it ever had in recorded history. These accounts often center on defining socioeconomic status in terms of college degree attainment. Older related analyses make distinctions between definitions of SES, and importantly demonstrate that SES divides play out differently across different areas of the country (distinguishing between voting pattern evolution in and outside the South, for example).

With some of these recent and older considerations in mind, I turned to the American National Elections Study, which allows for demographic breakdown of important political behavior metrics going back decades. This will serve as more of a quick, descriptive account of historical trends, and contextualization of the 2016 election year. The first section will compare voting and partisanship trends among whites of varying educational attainment. The second section will filter down to only whites without college educations, and look at the same political outcomes but separated by whether ANES respondents hail from a Census-defined Southern state or not.

1) Documenting the White Educational Divide

The below graph shows the progression of vote choice among whites from 1948/1952 to 2016, divided by whether the white ANES respondent’s highest level of education was a college degree or less than that.

Among college-educated whites, the general trend is toward greater vote choice for Democratic candidates and movement away from Republican candidates. The pattern starting in 1980 is clearest, as during that year 26 percent of college whites voted Democrat. That number has gradually climbed to 46 percent in 2016, which also is the same percentage of college-educated whites that voted Republican. Over the entire time span, the Republican decline is even steeper, dropping from a 72 percent share of the college white vote in 1952 to 46 percent in the last election. The same type of distinct trend is less obvious among non-college whites. From 1948 to 1976, the vote choice of this group fluctuates wildly, with blue swings also returning during Bill Clinton’s elections. The 2000 election starts a gradual bifurcation, as more non-college whites vote Republican and fewer vote Democrat. That culminates in 2016, when Donald Trump had a 30 point margin of victory among this group over Hillary Clinton.

The below graph takes a similar approach but tracks partisanship rather than vote choice over time.

Trends in partisanship exhibit greater stability across election years, but tell a similar story to the one above: an educational divide in political behavior grows over time, only in this case it proves more evident among the non-college-educated. After holding a partisan advantage over Republicans from 1952 to 1980, Democrats suffer from realignment that sees a steady non-college white flight from their party. 59 percent of this group identifies as Democrat in 1952, but only 35 percent does so in 2016. Interestingly, Republicans do not fully reap corresponding rewards from the Democrats’ losses in identification. Republicans gain 12 percent more non-college whites in 2016 compared to 1952, but so too does the “Independent” classification during this same period. College-educated whites, on the other hand, showcase much more stable aggregate partisanship over time, as the Republican advantage for this group hovers around 10 percentage points in each election year. This likely owes to this group’s greater political sophistication–a function of their greater education–which has been shown to produce more stable political preferences and predispositions (e.g. Zaller 1992; Gilens and Murakawa 2002).

College whites’ partisanship stability and non-college whites’ movement towards Republicans should beg an important question in a context where partisanship heavily shapes vote choice: how do Democrats stay competitive in elections? I won’t touch on this much here, but the short answer is that the increasing racial diversification of the electorate roughly grows in tandem with the white behavioral dynamics described above. Given many of these non-white entrants into the political sphere were much more disposed to favoring the Democratic Party, these developments together changed coalition compositions but preserved the electoral balance between the major parties.

2) A Regional Split among Low SES Whites

The above two graphs make clear that the movement and realignment in political behavior among whites is more pronounced among its non-college-educated population. Such greater year-to-year swings and over-time developments often translate to more consequential impacts on political events in the country. With this importance in mind, the below analysis cuts down to only evaluating whites without a college degree to better parse out historical trends in their political behavior. Namely, the same over-time outcomes are shown–starting with vote choice below, and partisanship later on–but broken up by region: respondents from the South and outside the South.

The above graph showing this breakdown for vote choice reflects an important claim made by Larry Bartels (2006) about a decade ago: most of the non-college white movement change in vote choice occurs among Southerners. It’s indeed clear that outside of the South there does not appear a strong, uniform movement in vote over time. (Though over the last three elections there is some steady movement towards Republican candidates and away from Democratic ones.) That contrasts from the story on the right-hand side of the graph, where save for the Clinton years, non-college whites in the South shift decidedly more Republican in voting behavior starting in the 1980 election. From 2000 to 2012, this Republican advantage stays constant. But the 2016 election ushered in an even greater divergence: Republican margin among Southern non-college whites grew from +37 in 2012 to +57 in 2016. That has further accentuated the regional split in vote: while Trump won non-college whites outside the South 55-39 in 2016, he won non-college white Southerners 76-19.

To round out this analysis, the below graph shows how partisanship trends among non-college whites break down by region.

The most astounding phenomenon that unfolds here is the precipitous decline in Democratic identification among non-college white Southerners. This group once self-identified as Democrat at a 79 percent rate back in 1952. In about every year thereafter, Democratic identification drops, plummeting down to 26 percent in 2016. Conversely, 36 percent more non-college white Southerners identify as Republican from the first to last point in this time period; 13 percent more also identify as Independent with no lean towards either party. Non-college whites living outside of the South also flee the Democratic Party, but not nearly to the same degree–from 1952 to 2016, Democratic identification drops 13 points from 53 to 40 percent, paling in comparison to the 53 point comparable drop in the South. This difference in rate of movement provides further (and more recent) evidence for one of Bartels’s (2006) conclusions. Changes in the non-college white vote–and white partisanship, as I introduce here–are certainly evident over time, but concentrate to a large degree among people in the South.

---

Note: For some years, and especially older ones, there are smaller weighted sample sizes for the above group breakdowns. I didn’t insert confidence intervals for each year-group-vote data point to avoid messy graphs, but the noisy nature of some of these estimates should be kept in mind (this imprecision might explain some of the more violent year-to-year swings, particularly earlier in the graph timelines, though partisan realignment around this time period also provides a compelling explanation). However, none of the weighted sample sizes for the year-demographic groups ever dipped below 100 (few reached that low anyway), so subgroup size problems are far from egregious.

Partisan Survey Responsiveness and Trump Approval in Consecutive Polls

Past findings regarding differential partisan nonresponse–driven by positive or negative news surrounding each major party–has largely come in the context of election seasons. Recently, I’ve checked whether this phenomenon extends to other salient public opinion trends, such as presidential approval, outside a campaign season. While lacking optimal (raw panel) data, I did find evidence that would tentatively confirm the main thrust of this past work–that variation in public opinion depends on the partisan makeup in the same poll (indicative of varying partisan responsiveness to surveys). Examining this relationship among pollsters who do and do not account for partisan selection processes in their weighting methodology sheds particularly convincing light on this dynamic. This specific aspect (the split between type of pollster) could imply that while this relationship could also be affected by individual level party identification shifts and resulting changes in party compositions, at the very least some of it has to stem from differential partisan nonresponse trends.

To test how robust of a finding this was, I wanted to check some alternative approaches for gauging the link between swings in partisan survey response and swings in the same poll outcome measures over time. One interesting alternative is to examine changes in partisan makeup and changes in outcomes of interest in consecutive polls from the same pollster. This builds off analysis of publicly available polls done by Gelman et al. (2016: 107) early in their paper regarding the same concept (see Figure 1(b)). Such an approach can prove helpful because it can better capture how movement in partisan makeup impacts movement in Trump approval. Comparing polls from the same pollster also could make for a better comparison (i.e. not needing to worry as much about how pollsters differ along other methodologies or approaches of theirs). This approach produced two new measures that I plot in the graph further below. Here are some details about these new variables (for any one poll conducted at time t).

1. Change in Net Republican = (Unweighted Republican %_t – Unweighted Democrat %_t) – (Unweighted Republican %_t-1 – Unweighted Democrat %_t-1)
2. Change in Net Trump Approval = (Trump Approve %_t – Trump Disapprove %_t) – (Trump Approve %_t-1 – Trump Disapprove %_t-1)

As with prior analyses, most of my data comes from HuffPost Pollster’s dataset of polls conducted from the start of Trump’s presidency through July 18th. Unlike before, I also searched for sample partisan composition–for polls that excluded this data in the available database–and manually recorded some of this excluded data from pollsters’ topline and crosstab pages. As a result, this made for a more comprehensive set of Trump approval data.

The below graph plots the relationship between the two aforementioned variables, distinguishing between pollsters who do and do not adjust for their sample’s partisan makeup (through party identification or past vote weights):

As the right-hand plot shows, a positive relationship emerges between consecutive poll shifts in partisan makeup and shifts in net Trump approval. Notably, the left-hand side contrasts with this result. Among individual pollsters who make some partisan adjustment, poll-to-poll changes in Trump evaluation they produce are largely uncorrelated with corresponding changes in their unweighted balance of partisans. It’s worth noting that the relationship’s strength among polls that don’t make any of these additional weighting adjustments is weaker than in my previous analysis (that did not track consecutive poll change). With that in mind, it still remains notable that 1) a positive relationship exists and 2) a clear difference in pollster type results. This alternative look provides additional evidence for the claim that swings in Trump approval are in part a byproduct of changes in partisan composition of polls–the latter of which speaks to a possible phenomenon of differential partisan nonresponse.

Below, I also show this same relationship between consecutive polls but among individual pollsters (limited to pollsters with more than three approval rating polls as of July 18th). Regression lines for the relationship between the two variables by pollster are once again plotted in each grid. Triangular points denote pollsters that adjust for their sample’s partisan makeup, while circular points represent pollsters that do not.

Small samples of polls abound in this graph, and so this should serve more as a qualitative look at a finer level (as well as to check whether certain pollsters disproportionately account for the relationship before). In a general sense, results from this graph confirm previous observations: unlike for pollsters that weight by party or past vote, pollsters that do not make adjustments for partisan sample makeup see their main outcome of interest–Trump approval–swing partly in conjunction with changes in how many partisans select into taking their polls. This is evidenced by the many positive relationships among different pollsters that don’t add special weights (circular dots) in light of the three pollsters–YouGov, ICITIZEN, and IBD/TIPP–that do make adjustments and see little relationship. From a qualitative look among survey houses with several polls during the Trump presidency, it seems as though Trump approval numbers produced by Politico/Morning Consult and even SurveyMonkey are most shaped by the partisan distributions of their samples.

The main takeaway here is a similar one from some past posts of mine. There are several ways to look at this phenomenon, but it does seem that differential partisan nonresponse plagues many pollsters in a way that suggests swings in Trump approval are partly sample artifacts, while pollsters who make adjustments for the partisan makeup of their polls tend to avoid such problems.

Short-Term and Long-Term Shifts in Individual Level Party Identification

Are certain trends in partisan public opinion tied more to changing party composition than is often assumed when looking at data? That’s the question that has been increasingly raised of late, particularly in light of developments in Republican public opinion. Stability of Trump approval among Republicans could be deceptive, for example, because Republicans who dislike him could have shed their party label, while those who continue to support him still identify as Republican and in turn leave a more “fanatical” bunch to make up Republican respondents.

These important endogeneity questions (e.g. uncertainty over the direction of the causal arrow between Republican identification and Trump support) take different forms. In some cases, it concerns the effect the candidacy and election of Trump has had on who chooses to identify as Republican–and the ill-equipped nature of cross-sectional surveys to address this potential change. I recently used the two panel waves before and after this past election as part of 2016 CCES data to approximate what kind of effect Trump’s election had on party movement (more notes on this Twitter thread). While there were statistically significant effects of race and ideology on the decision to either defect from or to the GOP from before to after the election, they weren’t large. So at least with that CCES data, the election itself did not seem to have too large of an impact on who chooses to identify as Republican.

The other form is more long-ranging. If there are longer-term changes in what type of people and demographic subgroups identify with each of the major parties, cross-sectional opinion data over several years cannot distinguish between actual opinion change and reshuffling party makeups. A time series showing Republicans changing their opinion on an issue from 2010 to 2017, for example, could actually be more tied to who chooses to identify as a Republican.

To try to gauge some of these longer term compositional shifts, as well as get a sense of the dynamics before Trump’s entry into the U.S. political scene, I took a look at 2010-2014 CCES panel data. The same 9,500 Americans were surveyed in 2010, 2012, and 2014 as part of this study, which allowed for tracking of individual level party identification shifts. The sizable sample in this data allowed for these shifts to then be evaluated along different demographic traits. I took a quick look and focused on two in particular–the first, age group, is shown below, while the second, education among whites, is addressed later.

Each column grid here divides Americans into the partisan groups with which they identified in 2010. The x-axis represents the percentage partisanship distribution of these groups when they were asked about their party in 2014. I break this trend up by four age groups in the above graph. The general takeaway from this result about individual level partisanship shifts is that the Democratic Party has become increasingly younger and the Republican Party has become increasingly older during this four-year pre-Trump span. While 92 percent of 18-44 aged Democrats in 2010 continued to identify as Democrat in 2014, fewer young Republicans in 2010 (86 percent) maintained Republican identification four years later.

Notably, the defection away from the Republican Party shrinks as age group gets older (looking from the top to bottom of the graph). The trend isn’t as perfectly gradual on the Democratic side, but the 12 percent of the oldest Democrats who defect four years later–eight percentage points more than Republicans in the comparable situation–offers a good indication of a partisan reorientation with respect to age. Moreover, pure Independents age 65 or older in 2010 shift in greater amounts to the GOP (25 percent) than to the Democratic Party (11 percent) when evaluated four years later.  Importantly, this individual level panel data approach can confirm an age-based partisanship shift that’s not tied up with increasing non-white shares of the youngest population (more on this here).

The second demographic trait by which I wanted to break down partisanship change was educational level among whites only. The graph below for this dynamic follows the same approach as the previous one, except four educational groups replace the four age groups.

Interestingly, partisanship change from 2010 to 2014–at least with this data–shows stability among people who identified as Republicans in 2010. The effects of outside shifts are still small but less stable. Movement among 2010 Independents in 2014 appears to make the GOP trend toward a population with lower formal education levels, as the three lower educational groups here enter the Republican Party more than the highest educational group does. On the other end, it’s clear the Democrat Party is growing more educated in terms of who it loses and who it gains as party members. A steady gradient exists among 2010 Democrats such that as educational level decreases, defection from the Democratic Party increases. 97 percent of post-graduate Democrats remain Democratic in 2014, while only 84 percent of 2010 Democrats with a high school education or less remain so. Additionally, the amount of 2010 Independents the Democratic Party gains in 2014 is positively correlated with their educational level. This trend with Democrats in particular is consistent with other accounts of a growing educational partisanship divide.

In sum, this type of data–at an individual level–can confirm that partisanship shifts observed in cross-sectional data have been in effect before Trump’s entry into American politics. If certain subgroup traits are correlated with certain public opinion expressions, then this becomes important for how we understand partisan public opinion change (or stability). Panel data that includes a 2016 wave would be of course the optimal type to answer a lot of the aforementioned endogeneity questions. Still, these results suggests party compositional shifts are certainly real. Helpful next steps in the future would track how the same respondents answer identical questions in 2012 as opposed to 2016, for example, to more rigorously evaluate the meaningfulness of cross-sectional time-series public opinion data.

Partisan Nonresponse and Obama Approval Polls

It seems that once you start looking for differential partisan nonresponse, it turns up in a lot of places. That’s the case not only for current and relatively recent data, but also–to a lesser degree–for data from the past decade, as I’ll cover in this post with President Barack Obama’s approval rating polls.

In my last post, I showed a fairly strong positive relationship has formed between the partisan makeup of a poll and Trump job assessment in approval ratings polls over the last few months. Most interestingly and indicative of partisan nonresponse bias effects, the relationship does appear for polls that make no effort to adjust for the partisan character of their sample in weighting methodology, but largely does not appear for polls that weight by party identification or past vote. The pattern among polls that don’t include this type of weight suggests that the partisan composition of a poll–rather than solely true opinion movement–factors substantially into determining Trump approval.

Hypothetical alternative explanations exist for this result. Polls vary in their sampling approaches, weighting schemes, and even how they report data, such as partisan distribution. These attributes in one way or another could possibly influence the relationship I observed. While I can’t account for everything, it’s worth noting that the statistically significant positive effect of partisan makeup on net Trump approval does hold up in a multivariate model controlling for other poll characteristics. Another explanation could entail an idea that individual party identification moves in tandem with presidential approval. Of course, this stands in stark contrast to the wealth of research demonstrating the stability of partisanship. Some research has shown there’s more individual level partisanship movement than widely assumed, but that’s not nearly enough to make this alternative plausible. (More importantly, for this to matter, there must be drastic real shifts in aggregate partisan distribution–not just individual level shifts that can cancel out–which clearly do not occur.)

Bivariate relationships–between partisan composition and public opinion that heavily involves party politics–can thus be very telling. Just as I did with Trump approval rating polls earlier, I wanted to check the party composition vs. net approval relationship, this time using approval rating polls of Obama during his presidency. Once again, I turned to HuffPost Pollster’s extensive database of polls that span the start of Obama’s presidency in 2009 to its end in 2017. From here I was able to record my independent variable (the difference in unweighted percentages of Democrats and Republicans) and my dependent variable (the difference in percentages approving and disapproving of Obama as president). For more detail on methods, and an important note about data collection (in paragraph 7), check my last post that covers very similar ground.

In the below graph, I plot this relationship. Polls that include some weighting adjustment for ideology or past vote choice appear on the left-hand side in red. There were very few non-YouGov pollsters that fell in this category, so for simplicity and the purposes of maintaining large enough samples of polls from individual survey houses, I only include YouGov for this classification. Polls that don’t include these type of weights–i.e., that don’t adjust for nonrandom partisan selection into polls–fall on the right-hand side of the graph in black.

Little relationship between partisan composition (a proxy for partisan nonresponse patterns) and net Obama approval exists among YouGov polls. Apart from a few polls that contain net even and pro-Republican makeups (that play a big role in turning the line of best fit slightly negative), the unweighted partisan distribution of YouGov polls stays fairly stable. On the other hand, among polls that don’t weight by ideology/past vote, a weak to moderate positive relationship (correlation coefficient of 0.29) forms between partisan makeup and Obama approval. The R-squared is small (0.08), though, especially compared to the same number but for Trump approval polls I found before (0.45).

Nevertheless, while not as strong, there does appear to be at least some effect of a poll’s partisan makeup on its ultimate outcome of interest, evaluation of a president. As noted before, this pattern suggests evidence of differential partisan nonresponse bias, wherein partisans non-randomly select into/out of polls in a way that makes results contingent on how many partisans on each side choose to take a poll. Notably, this differs to some degree from polls that try to account for this differential nonresponse activity (i.e. YouGov), which show less of a relationship. While not nearly as compelling as with Trump approval polls, it does seem that even during the Obama presidency with approval ratings, polls that don’t account for varying partisan willingness to take surveys suffer from bias.

Possible Evidence of Differential Partisan Nonresponse in Trump Approval Polls

Differential partisan nonresponse–the idea that survey response rates vary by partisanship–has received a lot of attention as of late (I had some thoughts on this subject a while ago too). Most notably, strong evidence exists that this specific pattern can explain much of the opinion movement seen in pre-election polling during the campaign period, and might be responsible for polling error in this past election. Nonresponse bias of any kind has been considered as a source for election polling error as well.

The differential willingness to partake in surveys among Democrats and Republicans in response to current events is more intense during an election season, but likely not confined to it. Politicization of public opinion polls (to some extent) and many other factors such as trends in social trust could mean that partisan nonresponse bias is becoming more widely entrenched. The current political mood of a country and favorable events for one’s political group may regulate survey response willingness for salient topics beyond vote intention, such as presidential approval of Donald Trump.

This phenomenon has important implications outside of just consideration for survey research practices. If the amount of Democrats or Republicans selecting into and out of polls shapes outcomes of interest, movement in public opinion become more illusory and an artifact of survey error. Corrections for this error, such as weighting to benchmarks for party identification or past vote distributions, seem promising but have remained far from embraced by the survey research community (in part, it depends on survey data types, such as panel or cross-sectional surveys).

In short, the questions over the severity of partisan nonresponse bias and how to correct for it are incredibly important but largely unsettled. The best current data to help speak to this issue lies in Trump approval rating polls. The Trump presidency has thus far been plagued by controversies scattered across his roughly half-year in office, introducing potential for partisan nonresponse. For example, at various points in the last few months, it’s very conceivable that Republicans heard bad news about their party’s president and became less willing to participate in surveys–likely about a topic unfavorable to them and their party–that were conducted around the time of this bad news.

To test this general idea, I compared the partisan composition of a poll and the level of Trump support in the same poll. I took data from HuffPost Pollster‘s database, which included all Trump approval polls conducted since his inauguration as president, and narrowed down to only look at polls that reported partisan breakdowns of Trump approval and the total number of partisans contained in a poll. Not all polls report partisan breakdowns and thus the Pollster data does not contain this data for several polls. Polls that report partisan breakdowns tend to be conducted more online than through live phone interviews. Besides this, it’s unlikely the polls with breakdowns I include and the polls without breakdowns I exclude systematically differ in many other ways and to the point that it would introduce serious bias (in a context where I want my results to speak to a phenomenon occurring in all polls).

For each poll–124 in total that included partisan subgroup information–I then calculated net Trump approval (the percentage approving of him minus the percentage disapproving of him in the aggregate) as well as the unweighted percentage of Democrats and Republicans in each poll. Using these latter pieces of data allowed me to create a partisan distribution metric: the percentage of a poll that was net Republican (Republican percentage of a poll minus Democratic percentage of a poll).

One note on this last variable: every pollster outside of Ipsos/Reuters in my data broke down Trump approval using a party identification measure that did not group Independents who leaned toward a party with that party. In most contexts, this offers a very deceptive picture of the partisanship of survey respondents (I explain why here). However, the “Republican % Net” variable I created should suffice as an adequate proxy for the partisan composition of a poll (the partisan distribution difference likely changes little on account of grouping or not grouping leaners).

In the below graph, I plot this partisan composition variable (net Republican percentage) against net Trump approval percentage. If Trump approval increases (decreases) as the number of Republicans in a poll increases (decreases), then this would roughly confirm a story of differential partisan nonresponse. I also add another wrinkle to this process in measuring the same relationship by breaking up the plot by whether a poll was done by a pollster that weights by party identification benchmarks (ICITIZEN and IBD/TIPP) and 2016 vote choice (YouGov, in partnership with either The Economist or HuffPost), or not. Adding this type of weight works to correct for differential partisan nonresponse, so comparing polls that do not include any correction to those who do provides the clearest picture.

In accordance with the presence of differential partisan nonresponse, the amount an unweighted poll sample is Republican is closely linked to Trump’s net approval rating in polls that do not weight by party or past vote. Specifically, the partisan difference variable explains 45 percent of the variation in Trump approval. As more Republicans select into taking a poll, the better the Trump approval rating becomes. Importantly, this would suggest that the key outcome of interest–how many Americans approve of Trump as president–hinges considerably on how many Republicans and Democrats choose to take a poll. This would break with the implicit assumption most people hold when evaluating polls that volatility in Trump approval ratings reflects actual change in people’s opinion of him. When Trump’s approval declines, for example, this might not indicate fewer Americans approving of him but rather fewer Republicans taking surveys–and the pollsters themselves doing nothing to correct for this bias.

But for the pollsters who do attempt to address partisan nonresponse bias–on the left side of the above graph–the same pattern does not hold. Because these pollsters use different methods to weight by party/past vote, I show the plotted relationship between partisan composition and Trump approval separately for the three survey houses (the two YouGov polls in red, IBD/TIPP in blue, and ICITIZEN in green). For all three of these pollsters that use unorthodox weighting methods, an equally strong relationship between partisan distribution of a poll and Trump net approval does not exist, and importantly not in the positive direction as seen in the right-hand side of the graph.

Adding weights for party ID or 2016 vote choice also seems to limit the variation in partisan composition from one poll to the next. Outside of one outlier, polls from YouGov all show a similar partisan distribution. For IBD/TIPP that’s less so the case, but for ICITIZEN the lesser variability is very clear. This variability aspect as well as comparing the plotted relationships before shed light on an important phenomenon: while pollsters that do not weight by party or past vote see a strong relationship between partisan distribution and Trump approval in their polls–indicative of differential partisan nonresponse effects–pollsters that do employ this weighting method seem to better overcome nonresponse bias problems (as expressed by the weak relationships in the left half of the graph).

It should be noted that all of these results materialize at a very crude, cross-sectional level. Panel data and studies, for example, would offer the most definitive account of partisan nonresponse trends. Nevertheless, it should remain notable that a strong relationship indicative of nonresponse bias still forms using this limited data. To round out this analysis, I also model Trump approval and account for other poll characteristics below using linear regression for only polls that do not weight by party/past vote. In the first column, I show the output for the bivariate regression illustrated in the plot–regressing Trump net approval percentage on Republican net percentage advantage in each poll. In models 2-4 I add in controls, such as an indicator for polls that use a registered voter universe relative to a baseline of all adults, and indicator for polls that use live phone surveys with online surveys as the base, and the number of respondents in a poll.

The first model shows that as the net Republican percentage in a poll moves one point up, Trump’s net approval percentage increases by 1.4 points. This positive statistically significant relationship is the same one expressed in the plot above. Adding in the various controls one by one in subsequent models decreases the effect size of the partisan composition variable on Trump approval a bit, but in each specification, the relationship remains positive and statistically significant.

There are of course other factors that I did not account for and others I could not measure, but the regression analysis still serves to bolster evidence behind the presence of this relationship–suggestive of partisan nonresponse bias effects–in surveys conducted by pollsters who don’t weight by party or past vote. These results of course should not imply that weighting by partisanship or past vote is the definite answer to solving the problem of differential partisan nonresponse. However, given the goal of polling remains to accurately capture opinion and its shifts, and in light of this considerable impact of a poll’s partisan makeup on a poll’s results, differential partisan nonresponse should be treated more seriously and new weighting methods considered as one option to address the issue.

---

7/19/17 edit: Some of this analysis was featured in a post on the Washington Post’s Monkey Cage blog, which you can find here.