3 ms·
> The polls were wrong, and 538 told us what would happen in that scenario. How do you define the mythical "standard polling error" though? They have historica
by JeremyNT 6y ago
> The polls were wrong, and 538 told us what would happen in that scenario.
How do you define the mythical "standard polling error" though? They have historical data, but polling techniques are always shifting.
Per the article:
> Suppose we’d included wider uncertainty intervals so the outcome was, say, within the 50% predictive interval. Fine. If we’d given Biden a 75% chance of winning and then he wins by a narrow margin, the forecast would look just fine and I’d be happier with our model. But the polls would still have messed up, it’s just that we would’ve better included the possibility of messing up in our model.
If presidential polls are consistently off by large margins, and 538 considers those margins to be unlikely every time, (which seems to be where we'll land this year as well as we did in 2016), then maybe it's not just the polls that are wrong, but the 538 assumptions about poll accuracy as well.
2016 on its own doesn't condemn the 538 methodology, but 2020 is starting to look like a trend. Yes, outlier events happen, but if they happen consistently... well, maybe your definition of "outlier" is incorrect. The sample size on presidential elections is small, and maybe there's just too much uncertainty for a model like 538's to end up being useful.
- mcphage 6y ago> 2016 on its own doesn't condemn the 538 methodology, but 2020 is starting to look like a trend. Yes, outlier events happen, but if they happen consistently... well, maybe your definition of "outlier" is incorrect. I get what you're saying, and I agree—but then, what is to be done? 538 doesn't do polls themselves, they just aggregate them. If the polls are incorrect, what do you replace them with? And how do you make that scientific?
- JeremyNT 6y ago> I get what you're saying, and I agree—but then, what is to be done? 538 doesn't do polls themselves, they just aggregate them. If the polls are incorrect, what do you replace them with? And how do you make that scientific? The most straightforward way to adjust these models, that Gelman mentions himself, is to just assume more uncertainty in the polls and assign wider uncertainty intervals to the models' final projections. It doesn't "fix" the polls but it acknowledges that they aren't as predictive as previously assumed. One of the comments suggests integrating betting market data in some way, which seems to make some intuitive sense based on how well they have performed relative to polls.
- taeric 6y agoIsn't that exactly the complaint against what 538 is doing? Literally upthread, "[the] issue with 538’s models is how wide their margins are."