Nate Cohn

Nate Cohn @Nate_Cohn

RT @kathrynw5: Wow, Beto really grew a beard since he dropped out of the race https://t.co/DDeuMI5IO8 — PolitiTweet.org

Nate Cohn @Nate_Cohn

@ForecasterEnten i kno u love it but these alerts are overkill — PolitiTweet.org

Nate Cohn @Nate_Cohn

@RyanDEnos in our last poll of WI, 52% of those without an age on the file were under age 30, 79% under age 45 — PolitiTweet.org

Nate Cohn @Nate_Cohn

@RyanDEnos yeah, though we poll off of the file so i don't really think of it as matching — PolitiTweet.org

Nate Cohn @Nate_Cohn

@RyanDEnos they are mainly young people fwiw — PolitiTweet.org

Nate Cohn @Nate_Cohn

@RyanDEnos yes, it sucks — PolitiTweet.org

Nate Cohn @Nate_Cohn

@kwcollins that would be my expectation as well, though i'd say the change in variance is more on the online side, which is now all but totally solid and previously had some variability — PolitiTweet.org

Nate Cohn @Nate_Cohn

@kwcollins from the standpoint of methods or variance in result? — PolitiTweet.org

Nate Cohn @Nate_Cohn

If I have any good news, it's that this could be less of an issue in the heart of the horse race, when most online pollsters stay out of publishing hard election results and live interview pollsters will push in their resources. But it will remain troubling either way. — PolitiTweet.org

Nate Cohn @Nate_Cohn

If the online polls and live interview polls act in uncorrelated ways, a simple house effect adjustment doesn't do the job. We could split the difference, but there's little reason to assume the average is any likelier than one or the other, and online dominates w volume anyway — PolitiTweet.org

Nate Cohn @Nate_Cohn

The bottom-line: this is a challenge for analysts and the industry. The core tension is that the transition to online polling is happening, but the online polls may still (not necessarily!) be immature. Live interview polls are dying, but often considered the 'gold standard' — PolitiTweet.org

Nate Cohn @Nate_Cohn

It is hard to evaluate this without more data, especially for the online pollsters who don't publish many hard election results, just national polls. In Morning Consult, their '50 state estimates' lead me to think the issue could be specific to rural whites (NE!) but idk. — PolitiTweet.org

Nate Cohn @Nate_Cohn

Alternately, one could argue this is the dying breath of live interview polls, which have lower and lower response rates. Here, online polls would say there are very few changing opinions at this point, and the live polls are just catching varying differential response — PolitiTweet.org

Nate Cohn @Nate_Cohn

But there are worse scenarios, too. Perhaps panel conditioning, for instance, has taken a toll on the data being used on online pollsters. Basically, the millions of online panelists would have developed an independent set of opinions and trends after years of taking online polls — PolitiTweet.org

Nate Cohn @Nate_Cohn

You can certainly see that in the Democratic primary race. These online pollsters have very different takes on the race, and very few of them seem to simply be the live interview average without revision. — PolitiTweet.org

Nate Cohn @Nate_Cohn

I'd note, FWIW, that the Civiqs tracker, which isn't included here as it is not in the FiveThirtyEight database, does seem to track more with the live interview polls. Am I sure that if we had 5 more big, online pollsters that this pattern would still hold? No, not really — PolitiTweet.org

Nate Cohn @Nate_Cohn

A final possibility, and one that maybe you should root for, is that it's basically a coincidence--we have only a small number of diverse, idiosyncratic online pollsters. They have their own patterns, and can't be clumped like the live RDD polls, which are fundamentally similar. — PolitiTweet.org

Nate Cohn @Nate_Cohn

Another possibility is a mode effect, where people respond differently online. But it wouldn't really explain a lack of variance, just bias. It made more sense in '17, though even then it was unproven and the idea that it flipped is tough to stomach. — PolitiTweet.org

Nate Cohn @Nate_Cohn

Another explanation is noise: the low volume of live interview polls means that our estimate of live interview polls is uncertain at any time (I don't think we can be sure that WaPo, Pew couldn't say Trump at 38/58 tomorrow). But the pattern has lasted long enough for me. — PolitiTweet.org

Nate Cohn @Nate_Cohn

So what's going on? Well there are some methodological explanations, certainly on individual online pollsters (YouGov's heavier weights should impose less volatility, rightly or wrongly). But most online polls aren't like YouGov and it holds without them — PolitiTweet.org

Nate Cohn @Nate_Cohn

Notice: --online has basically been higher than live for a year, with online showing no variability --the recent and abrupt 'jump' in the president's approval is exclusively due on the live interview side (we often go weeks without them, so it shows up abruptly here) — PolitiTweet.org

Nate Cohn @Nate_Cohn

Notice: --blue has basically been higher than red for a year, with no variability --the recent and abrupt 'jump' in the president's approval is basically exclusively due to an infusion of live interview polls (we often go weeks without them, so it shows up abruptly here) — PolitiTweet.org

Nate Cohn @Nate_Cohn

The live interview polls are harder to measure, overall or individually. There are fewer of them, so the estimate is more volatile. But we have more than enough recent polls to say that it's not in the same place as online and better for Trump (online = blue; live = red) https://t.co/i3xBZkhY9h — PolitiTweet.org

Nate Cohn @Nate_Cohn

As a result, a Morning Consult house effect calculation now basically goes the wrong way. This is not a critique of 538 or House Effects more generally, and I don't think it messes up the averages over all, but it is a great illustration of how much the relationship has changed https://t.co/WgMV5NYSqn — PolitiTweet.org

Nate Cohn @Nate_Cohn

We have enough online data to evaluate individual pollsters, and they don't all act the same. YouGov is basically always unflinching; Morning Consult has an outright counter-cyclical trend: it was Trump's best pollster in '17, now it's his worse, and it's fairly stable in 2019. — PolitiTweet.org

Nate Cohn @Nate_Cohn

Though the chart isn't labeled properly, this is the president's rating among online-only polls (all adjusted to RVs). There's no 2019 variance, and it places the president lower than he was in 2018 https://t.co/5twnjRitY4 — PolitiTweet.org

Nate Cohn @Nate_Cohn

Now, live interview polls show the president's approval rating back up. And there are *a lot* of them at this point showing the increase, enough to move the FiveThirtyEight average to basically its all-time high. The online polls though? Not much of any movement — PolitiTweet.org

Nate Cohn @Nate_Cohn

This change in relative standing proved to be somewhat fleeting. The president's ratings slumped in the live interview polls in August, and dropped a bit more in the aftermath of the impeachment revelations. That brought them back to the same spot. But online polls never budged — PolitiTweet.org

Nate Cohn @Nate_Cohn

By mid 2019, live interview polls showed the president's ratings *higher* than online polls. Weirder still: they had recorded an increase in the president's rating, while online polls showed the president's approval totally unchanged or slightly down. — PolitiTweet.org

Nate Cohn @Nate_Cohn

In the early days of the Trump presidency, the live interview polls and the online polls tracked together very nicely, except that the online polls were a couple of points better for Trump--supporting various theories of shy Trumpers and so on. Not anymore. — PolitiTweet.org