What Democrats Could Lose With Their Left Turn (New York Times/The Upshot)

Over at The Upshot (New York Times), I wrote about a recent survey experiment that I ran. I randomly exposed people to a short summary of recent Democratic debates—and the various left-leaning proposed policies—or other unrelated political content, and measured various reactions regarding the 2020 election. Most notably, I find evidence of backlash among Independents in their intended vote choice, moving against the eventual Democrat nominee after they read about the sharp left turn taken by current Democratic candidates. In this post, I wanted to record details on the survey experimental design and methods.

Below is the instrument (parts in brackets were not included). Respondents were randomly assigned to either the treatment or control condition. All vignette text was drawn directly from real world news articles and adapted lightly. Specifically, I took from the New York Times, The New Yorker, the Los Angeles Times, and New York Magazine to create the treatment (borrowing from all four) and took from McClatchy to create the control.

[Shown separately as a first screen]

The next screen will display information from a recent news article. Please read it carefully and then answer some questions that follow.

[Treatment]

Primary debates showcase leftward turn of Democratic Party

The first few debates of the Democratic primary season have illustrated how far to the left the party has moved.

Many candidates supported decriminalizing illegal border crossing and expanding undocumented immigrants’ access to government services.

Others called for replacing a private health insurance with a government-run health system, and establishing free public college for all children from working-class families.

[Control]

The key dates on the road to the White House in 2020

As the Democratic primary heats up, it’s time to look ahead to the key dates during the 2020 campaign.

The Democratic nominating process officially begins in Iowa and New Hampshire.

Next up are the Nevada caucuses, South Carolina, and Super Tuesday.

Each party holds its national conventions in July and August.

Presidential debates are held in the fall.

The general election takes place November 3^rd.

[Outcomes]

Who will you vote in the November 2020 general election for president?

• Donald Trump
• The Democratic Party nominee
• Other
• Not sure
• I will not vote

Would you consider voting for the eventual Democratic nominee for president in the 2020 election?

• I would strongly consider it
• I would maybe consider
• I would likely not consider it
• I would never consider it

Are you motivated FOR or AGAINST the eventual Democratic nominee for president in 2020?

• Strongly motivated to vote and campaign FOR the candidate
• Somewhat motivated to vote and campaign FOR the candidate
• Neither motivated for nor against the candidate
• Somewhat motivated to vote and campaign AGAINST the candidate
• Strongly motivated to vote and campaign AGAINST the candidate

Note: A quasi-manipulation check question—about how liberal/conservative respondents rated the average ideology of the 2020 Democratic primary candidates—was not included in the article. This was done to keep the article more concise and focused, and also because it was a bit esoteric for a general audience. For Democrats and Independents (but curiously not for Republicans), the treatment had positive effects in making perceived ideology of the Democratic field more liberal, though only at marginally statistically significant levels.

Notes on data cleaning and analysis:

• A handful of respondents reported implausibly old ages on the Civis online panel (e.g. nine said they were born in 1910, i.e. they were 109 years old). Given this behavior among a very small subset of respondents, and in order be cautious, I restricted my sample to respondents who reported as old a birth year as 1936 (i.e. at most 83 years old). This removed 38 respondents. Results are robust to the inclusion of these individuals.
• 2020 vote intention was represented as a two-party choice, as the balance of votes for the Democratic and Republican options (excluding non-major party ones) is what most closely corresponds to the outcome of the election. Thus, a selection of “The Democratic Party nominee” took on a value of 1 and “Donald Trump” took on a value of 0.
• The two other outcomes were used turned into numeric variables and scaled to run from 0-1 (where 1 was “I would strongly consider it” on the second outcome and “Strongly motivated to vote and campaign FOR the candidate” on the third outcome).
• I use OLS regression to estimate treatment effects on all three outcomes, with robust standard errors (using lm_robust() from the estimatr package in R).
• I include one control variable throughout the analysis: 2016 vote recall, which was asked earlier in the survey, and is made up of three categories—Hillary Clinton vote choice, Trump vote choice, or other/didn’t vote. Even though the treatment and control were not too unbalanced along 2016 vote choice, past vote and intended 2020 vote choice are so correlated that even slight imbalances make a big difference. This makes it important to adjust for this pretreatment covariate when modeling 2020 vote, a decision that was also strongly recommended by those who coordinated my study at Civis.

---

More discussion here.

Advertisement