5 ms·

> If per-state error isn't normally distributed, that's evidence of bias, or bad polling. No! Assuming the per-state error would be normally distributed in so

> If per-state error isn't normally distributed, that's evidence of bias, or bad polling. No! Assuming the per-state error would be normally distributed in some neutral world is making huge assumptions about the nature of the electorate, polling, and the correlations of errors between states, you can't do that! You would specifically /not/ expect per-state error to be evenly distributed because the nature of the error would have similar impacts on similar populations and there are similar populations of people that live in different states in differing numbers. You should review the literature about the nature of the (fairly small) polling misses that impacted the swing states and thus disproportionately the outcome in the 2016 election. You will probably find it interesting.

listenallyall 6y ago
Yes! There are unavoidable, expected, sampling errors which are, by definition, random. That's why valid, trusted polls calculate a confidence interval instead of a single discrete result. Other types of "errors" -- election results that repeatedly fall outside the confidence interval, or are consistently on only one side of the mean -- only arise when the poll is flawed for some reason. Maybe you relied on landlines only, maybe you spoke with too many men, or too many young people, asked bad questions, miscalculated "likely voter," whatever. Accurate, valid, trusted polls don't have these flaws, the ONLY errors are small, random, expected sampling errors.
- Karrot_Kream 6y ago
  > That's why valid, trusted polls calculate a confidence interval instead of a single discrete result. That is what each of these statistical models did, yes. And the actual outcomes fell into these confidence intervals. > Other types of "errors" -- election results that repeatedly fall outside the confidence interval, or are consistently on only one side of the mean -- only arise when the poll is flawed for some reason. Or the model was inaccurate. Perhaps the priors were too specific. Perhaps the data was missing, misrecorded, not tabulated properly, who knows. Again, the results fell within the CI of most models, the problem was simply that the result fell too close to the mean for most statisticians' comfort.
  - listenallyall 6y ago
    Landed within the confidence interval? Are you kidding? CI is generally 2-4 points in these election polls. - Texas: 538 said Trump +1.0, actually won by 6 - Ohio: Trump +0.4, won by 8 - Iowa: Trump +1.4, won by 8 - Florida: Trump -2.5, won by 3 - Penn.: Trump -4.6, lost by 0.5 - Nevada: Trump -4.9, lost by 2 - Wisconsin: Trump -7.9, lost by 0.6 https://en.wikipedia.org/wiki/Statewide_opinion_polling_for_the_2020_United_States_presidential_election https://en.wikipedia.org/wiki/Statewide_opinion_polling_for_... https://www.nytimes.com/interactive/2020/11/03/us/elections/results-president.html https://www.nytimes.com/interactive/2020/11/03/us/elections/...
    - Karrot_Kream 6y ago
      > Landed within the confidence interval? Are you kidding? CI is generally 2-4 points in these election polls. The models and their data are public. The 538 model predicted an 80%CI of electoral votes for Biden as: 267-419, with the CI centered around 348.49 EVs. That means that Biden had an 80% chance of landing in the above confidence interval. Things seem to be shaking out to Biden winning with 297 EVs. Notice that this falls squarely within the CI of the model, but much further from the median of the CI than expected. So yes, the results fell within the CI. Drilling into Florida specifically (simply because I've been playing around with Florida's data), the 538 model predicts an 80%CI of Biden winning 47.55%-54.19% of the vote. Biden lost Florida, and received 47.8% of the vote. Again, note that this is on the left side of this CI but still within it. The 538 model was correct, the actual results just resided in its left tail.
      - listenallyall 6y ago
        Dude, you're gaslighting by using the national results as evidence instead of the individual states, which is what this has always been about since my original comment. Nearly every consequential state fell at, or beyond, the tail end of 538's confidence interval (BTW, who uses 80%? and not 90-95%?), on the same side. A bit closer to the mean in AZ and GA but same side, over-estimating Biden's margin of victory. Deny it all you want, gaslight, cover your eyes, whatever -- but clear, convincing, overwhelming evidence of a systematic flaw or bias in the underlying polls is right there in front of you. Many political handicappers had predicted that the Democrats would pick up three to 15 seats, growing their 232-to-197 majority https://www.washingtonpost.com/politics/house-races/2020/11/02/9823d974-1d64-11eb-b532-05c751cd5dc2_story.html https://www.washingtonpost.com/politics/house-races/2020/11/... Entering Election Day, forecasters projected Democrats would gain House seats and challenge for the Senate majority. https://www.cnbc.com/2020/11/05/2020-election-results-democrats-to-lose-house-seats-but-keep-majority.html https://www.cnbc.com/2020/11/05/2020-election-results-democr... Most nonpartisan handicappers had long since predicted that Democrats were very likely to win the majority on November 3. "Democrats remain the clear favorites to take back the Senate with just days to go until Election Day," wrote the Cook Political Report's Senate editor Jessica Taylor on October 29. https://www.cnn.com/2020/11/04/politics/2020-election-senate-majority-gop/index.html https://www.cnn.com/2020/11/04/politics/2020-election-senate...
- dragonwriter 6y ago
  > Accurate, valid, trusted polls don't have these flaw Yes, they do. Because (among many other reasons) humans have a choice whether or not to respond, you can't do an ideal random sample subject to only sampling error for a poll. All polls have non-sampling error on top of sampling error, it is impossible not to.
  - listenallyall 6y ago
    when polls don't match up with reality, as they didn't in 2016, the pollsters have a responsibility to re-calibrate the way they conduct the poll. Ask different questions, find new ways of obtaining respondents from all demographics, adjust raw data, etc. A professional pollster doesn't just get to say, hey, some people didn't want to talk to me ¯\_(ツ)_/¯
    - dragonwriter 6y ago
      > when polls don't match up with reality, as they didn't in 2016, the pollsters have a responsibility to re-calibrate the way they conduct the poll. Pollsters do that for continuously, and there were definite recalibrations in the wake of 2016. OTOH, the conditions which produce non-sampling errors aren't static, and it's impossible to reliably even measure the aggregate of non-sampling error in any particular event (because sampling error exists, and while it's statistical distribution can be computed the actual error attributable to it in a by particular event can't be, so you never no how much actual error is due to non-sampling error much less any particular source of non-sampling error.)
- JamisonM 6y ago
  "There are unavoidable, expected, sampling errors which are, by definition, random." This is false, if you think sampling errors are "by definition random" you don't understand polling. Which is fine, just accept that you don't and dig into the literature or move on.

listenallyall 6y ago

Yes! There are unavoidable, expected, sampling errors which are, by definition, random. That's why valid, trusted polls calculate a confidence interval instead of a single discrete result. Other types of "errors" -- election results that repeatedly fall outside the confidence interval, or are consistently on only one side of the mean -- only arise when the poll is flawed for some reason. Maybe you relied on landlines only, maybe you spoke with too many men, or too many young people, asked bad questions, miscalculated "likely voter," whatever. Accurate, valid, trusted polls don't have these flaws, the ONLY errors are small, random, expected sampling errors.

Karrot_Kream 6y ago
> That's why valid, trusted polls calculate a confidence interval instead of a single discrete result. That is what each of these statistical models did, yes. And the actual outcomes fell into these confidence intervals. > Other types of "errors" -- election results that repeatedly fall outside the confidence interval, or are consistently on only one side of the mean -- only arise when the poll is flawed for some reason. Or the model was inaccurate. Perhaps the priors were too specific. Perhaps the data was missing, misrecorded, not tabulated properly, who knows. Again, the results fell within the CI of most models, the problem was simply that the result fell too close to the mean for most statisticians' comfort.
- listenallyall 6y ago
  Landed within the confidence interval? Are you kidding? CI is generally 2-4 points in these election polls. - Texas: 538 said Trump +1.0, actually won by 6 - Ohio: Trump +0.4, won by 8 - Iowa: Trump +1.4, won by 8 - Florida: Trump -2.5, won by 3 - Penn.: Trump -4.6, lost by 0.5 - Nevada: Trump -4.9, lost by 2 - Wisconsin: Trump -7.9, lost by 0.6 https://en.wikipedia.org/wiki/Statewide_opinion_polling_for_the_2020_United_States_presidential_election https://en.wikipedia.org/wiki/Statewide_opinion_polling_for_... https://www.nytimes.com/interactive/2020/11/03/us/elections/results-president.html https://www.nytimes.com/interactive/2020/11/03/us/elections/...
  - Karrot_Kream 6y ago
    > Landed within the confidence interval? Are you kidding? CI is generally 2-4 points in these election polls. The models and their data are public. The 538 model predicted an 80%CI of electoral votes for Biden as: 267-419, with the CI centered around 348.49 EVs. That means that Biden had an 80% chance of landing in the above confidence interval. Things seem to be shaking out to Biden winning with 297 EVs. Notice that this falls squarely within the CI of the model, but much further from the median of the CI than expected. So yes, the results fell within the CI. Drilling into Florida specifically (simply because I've been playing around with Florida's data), the 538 model predicts an 80%CI of Biden winning 47.55%-54.19% of the vote. Biden lost Florida, and received 47.8% of the vote. Again, note that this is on the left side of this CI but still within it. The 538 model was correct, the actual results just resided in its left tail.
    - listenallyall 6y ago
      Dude, you're gaslighting by using the national results as evidence instead of the individual states, which is what this has always been about since my original comment. Nearly every consequential state fell at, or beyond, the tail end of 538's confidence interval (BTW, who uses 80%? and not 90-95%?), on the same side. A bit closer to the mean in AZ and GA but same side, over-estimating Biden's margin of victory. Deny it all you want, gaslight, cover your eyes, whatever -- but clear, convincing, overwhelming evidence of a systematic flaw or bias in the underlying polls is right there in front of you. Many political handicappers had predicted that the Democrats would pick up three to 15 seats, growing their 232-to-197 majority https://www.washingtonpost.com/politics/house-races/2020/11/02/9823d974-1d64-11eb-b532-05c751cd5dc2_story.html https://www.washingtonpost.com/politics/house-races/2020/11/... Entering Election Day, forecasters projected Democrats would gain House seats and challenge for the Senate majority. https://www.cnbc.com/2020/11/05/2020-election-results-democrats-to-lose-house-seats-but-keep-majority.html https://www.cnbc.com/2020/11/05/2020-election-results-democr... Most nonpartisan handicappers had long since predicted that Democrats were very likely to win the majority on November 3. "Democrats remain the clear favorites to take back the Senate with just days to go until Election Day," wrote the Cook Political Report's Senate editor Jessica Taylor on October 29. https://www.cnn.com/2020/11/04/politics/2020-election-senate-majority-gop/index.html https://www.cnn.com/2020/11/04/politics/2020-election-senate...
dragonwriter 6y ago
> Accurate, valid, trusted polls don't have these flaw Yes, they do. Because (among many other reasons) humans have a choice whether or not to respond, you can't do an ideal random sample subject to only sampling error for a poll. All polls have non-sampling error on top of sampling error, it is impossible not to.
- listenallyall 6y ago
  when polls don't match up with reality, as they didn't in 2016, the pollsters have a responsibility to re-calibrate the way they conduct the poll. Ask different questions, find new ways of obtaining respondents from all demographics, adjust raw data, etc. A professional pollster doesn't just get to say, hey, some people didn't want to talk to me ¯\_(ツ)_/¯
  - dragonwriter 6y ago
    > when polls don't match up with reality, as they didn't in 2016, the pollsters have a responsibility to re-calibrate the way they conduct the poll. Pollsters do that for continuously, and there were definite recalibrations in the wake of 2016. OTOH, the conditions which produce non-sampling errors aren't static, and it's impossible to reliably even measure the aggregate of non-sampling error in any particular event (because sampling error exists, and while it's statistical distribution can be computed the actual error attributable to it in a by particular event can't be, so you never no how much actual error is due to non-sampling error much less any particular source of non-sampling error.)
JamisonM 6y ago
"There are unavoidable, expected, sampling errors which are, by definition, random." This is false, if you think sampling errors are "by definition random" you don't understand polling. Which is fine, just accept that you don't and dig into the literature or move on.