Clinton vs. Trump: Differences by Survey Mode

Back in early January in the run-up to the first primaries in 2016, the debate over what different type of surveys signified hit a high pitch. Highlighting the divide between traditional live interview telephone polling and anonymous online survey-taking, the advantage held by Donald Trump in national and state polls was called into question. Outperforming in online formats led many–including myself–to believe that his actual level of support was getting spuriously inflated. After Iowa, that notion became much less convincing in light of Trump’s successes.

Now, this niche topic concerning survey mode has returned, but has gained attention from the broader public:

As a response to Trump’s precipitous fall in both state and national polls, his support network has responded by pointing to a supposed “hidden vote” lurking in the electorate. The claim directly ties in what different types of surveys are telling us: a Trump advisor says that this unaccounted advantage of Trump’s lies in online polls, where she claims he does better than in live interview polls, and that this is a better indication of the state of the race since online polling better approximates the act of voting (done anonymously).

To start, it’s important to note this alludes to a crucial dynamic that plagues surveys in which the interviewee directly interacts with a live person interviewer: social desirability bias. People may be less willing to reveal certain opinions that are less socially acceptable, and thus undermine the objective of opinion research of accurately capturing public attitudes. During several points during this election cycle, this issue’s relation to Trump’s polling numbers has received insightful and legitimate consideration.

But is the current claim coming out of Trump’s camp valid? To try to find any difference by survey type in Trump support–and see if live phone interviews substantially undersold Trump’s position relative to online surveys–I examined the margin of Hillary Clinton’s lead in live phone and internet polls over the last year.

Source: HuffPost Pollster.

Note: This data is from polls that only test head to head matchups between Clinton and Trump, which exclude mention of third party candidate names. 

The dotted line in the above graph approximates Clinton’s margin in live phone surveys, and the solid line does so for internet polls. While Clinton consistently leads Trump by a larger margin in phone interviews than in online ones throughout the time frame, the difference–maybe around a percentage point–is far from anything really substantive. That small gap in survey modes generally stays constant going back to general election trial heats conducted in late 2015 as well. Simply put, in the context of a Trump campaign in serious polling peril, pointing to potential survey mode effects may temporarily divert attention, but–at this point–seems very unlikely to have a discernible effect come November.

Advertisement