3 ms·

Thus far it actually seems to be due different response rates between Republican versus Democratic voters. Democrats seem more likely to respond to pollsters.

Thus far it actually seems to be due different response rates between Republican versus Democratic voters. Democrats seem more likely to respond to pollsters. Of course pollsters already know this, so they weight the samples to reflect underlying demographics, but it gets tricky to do this accurately if your surveying response rate is 1% to 2%. And of course... the forecast modelers know this. Which is what Gelman discusses in his post. Given that they now have 2016, and 2020 as examples of presidential polling errors, modelers need to shift their priors in a number of ways to account for what looks like a persistent undersampling of GOP voters.

dragonwriter 6y ago
> Of course pollsters already know this, so they weight the samples to reflect underlying demographics That doesn't actually help unless the difference in response rate is explained by the underlying demographics. If it is correlated with preferences on the issues polled even controlling for non-preference demographics, it's not going to be correctable by that means, nor is it going to be measurable so that it can be corrected. OTOH, if it is a long-term consistent effect tied to partisan presidential voting preferences across election (and not, e.g., tied to the particular transitory trends, or evolving significantly over time) you could, as a poll-based forecaster, correct for the effect without measuring the correlation in preference and response rates by adjusting how you draw forecasts from polling data by correctingnbased on empirical election results and their relation to polls.
JeremyNT 6y ago
> And of course... the forecast modelers know this. Which is what Gelman discusses in his post. Given that they now have 2016, and 2020 as examples of presidential polling errors, modelers need to shift their priors in a number of ways to account for what looks like a persistent undersampling of GOP voters. In the article, he mentions the simplest and probably most correct way to do this: reduce the confidence these models have in the accuracy of the polling. The top line number he threw out as feeling "good" about was 75% for a Biden win. As it is, it's becoming difficult to find any value in these models because the events they represent as outliers seem to be consistently occurring. On the other hand, if they adjust to more accurately reflect the uncertainty in the the polls, eventually everything starts looking close to a coin toss - in which case they have no practical value, either.

dragonwriter 6y ago

> Of course pollsters already know this, so they weight the samples to reflect underlying demographics That doesn't actually help unless the difference in response rate is explained by the underlying demographics. If it is correlated with preferences on the issues polled even controlling for non-preference demographics, it's not going to be correctable by that means, nor is it going to be measurable so that it can be corrected. OTOH, if it is a long-term consistent effect tied to partisan presidential voting preferences across election (and not, e.g., tied to the particular transitory trends, or evolving significantly over time) you could, as a poll-based forecaster, correct for the effect without measuring the correlation in preference and response rates by adjusting how you draw forecasts from polling data by correctingnbased on empirical election results and their relation to polls.

JeremyNT 6y ago

> And of course... the forecast modelers know this. Which is what Gelman discusses in his post. Given that they now have 2016, and 2020 as examples of presidential polling errors, modelers need to shift their priors in a number of ways to account for what looks like a persistent undersampling of GOP voters. In the article, he mentions the simplest and probably most correct way to do this: reduce the confidence these models have in the accuracy of the polling. The top line number he threw out as feeling "good" about was 75% for a Biden win. As it is, it's becoming difficult to find any value in these models because the events they represent as outliers seem to be consistently occurring. On the other hand, if they adjust to more accurately reflect the uncertainty in the the polls, eventually everything starts looking close to a coin toss - in which case they have no practical value, either.