4 ms·

Well for one, 538 gave Trump 10% at the lowest, which is quite a bit better than 4% by CU. Also, putting Florida aside, everything else so far has been well wit

Well for one, 538 gave Trump 10% at the lowest, which is quite a bit better than 4% by CU. Also, putting Florida aside, everything else so far has been well within the error margin, even if the overall mean is 3-4 points shifted towards Trump. I'm not an expert, but that's the whole point of confidence intervals. Polls will never be accurate, but it's about using the right uncertainty, and also mixing uncertainties properly. EDIT: Let's look at a few of the tipping point results from 538 [1] Florida: 50.9/48.4 -> 47.8/51.2 (2.8 diff) [99% reporting] Arizona: 50.7/48.1 -> 50.7/47.9 (0.1 diff) [86% reporting] Texas: 48.8/50.3 -> 46.4/52.2 (1.9 diff) [85% reporting] Georgia: 50.1/49.2 -> 49.1/49.7 (0.7 diff) [98% reporting] Minnesota: 53.7/44.6 -> 52.6/45.4 (0.9 diff) [99% reporting] It's also worth noting that a lot of these are still at 85-95 percent reporting, so it's not over, but yes I don't see anything that's very off, even Florida that was so off was only 3 points off. I do consider 3 points to be well within the toss-up category. [1] https://projects.fivethirtyeight.com/2020-election-forecast/ https://projects.fivethirtyeight.com/2020-election-forecast/ EDIT2: I think the main thing that surprised people is how in some places, mail-in votes came first, whereas in some other places, mail-in votes are coming much later. This really skews the early data but as things settle, everything is coming much closer to the predictions. But this is something 538 and others have been warning about since May or earlier.

ars 6y ago
> but that's the whole point of confidence intervals. That would make sense if some parts of the country had errors one way, and others another way. i.e. random error. But if every part of the country has an error, all in the same direction, then there's something larger going on.
- dragonwriter 6y ago
  > That would make sense if some parts of the country had errors one way, and others another way. i.e. random error. As Silver pointed out in 2016 (and why his model then gave Trump a much better chance than the NYTimes model), historically, when election outcomes differ from polls in either direction, the polling error between states is correlated, not independent. That is, it is, to all evidence, a historical fact that poll-based predictions must account for that within election cycles, there is systematic polling error, even if the direction and magnitude of that error is not consistent between cycles.
- akersten 6y ago
  > i.e. random error. Did they say that their confidence interval was confidence against random error? I don't think that's a requirement of a confidence interval, I think it's certainly valid to encode e.g. the chance of "shy Trump" respondents in a confidence interval.
- slg 6y ago
  The results of state elections aren't independent. The problems that cause most errors are going to exist in multiple states and cause the errors to be correlated.
  - thaumasiotes 6y ago
    But something that causes the results everywhere to be off in the same direction isn't the kind of error you say "well, that's why you don't make point predictions" about. It's a factor that has a significant, predictable influence on the phenomenon you're studying; if it's not in your model, that means you screwed up when making the model. The whole point of the model is to account explicitly for such factors.
- esrauch 6y ago
  I don't think it's expecyed the errors would be random: that the polling errors are correlated and not random is something that Nate Silver talks about at length as part of the key interpretation error that others made in 2016.
- Retric 6y ago
  No that says nothing about the polling just your model. 538 had an interesting page where picking a state would influence the model and change what other states where predicted to do. But I can’t seem to find it now. Figuring out the dependencies between demographics is how you come up with a useful model rather than just simulating everything independently based on raw polls numbers. Aka Michigan and Wisconsin have a lot in common where Florida is influenced by wildly different racial, demographic, and economic factors.
  - ehsankia 6y ago
    https://projects.fivethirtyeight.com/trump-biden-election-map/ https://projects.fivethirtyeight.com/trump-biden-election-ma...
    - Retric 6y ago
      Thanks, that’s what I was looking for. Back to my point, click Trump for Florida (which was considered a 31% chance) and the prediction looks a lot like our current map.
    - kgwgk 6y ago
      There is something similar for the Economist model: https://www.ricardofernholz.com/election/ https://www.ricardofernholz.com/election/
- SamBam 6y ago
  That's not their method at all. They're very insistent on the fact that voting shifts or errors between states are generally highly correlated. That's exactly why their probability forecasts tend to be more conservative than other forecasts. If you see a whole lot of possible paths for a candidate, and assume the errors are random, then that candidate will almost certainly win. If you know they might be correlated, then you know that a huge number of paths may shift at once.
- deleted 6y ago
  [deleted]
ttesttom 6y ago
This leaves out the states that originally tipped the election to trump last time due to polling error of 4-7pts. Really its big polling errors in these states. The rest of the map doesn't look as bad. Popular vote also most likely off by estimated 4pts when all ballots counted, which isn't terrible. Diff on final 538 -> current results Wisc: ~8pt diff Michigan: 4-6pt diff PA: still counting but most likely 2-5pts off
- ehsankia 6y ago
  Not sure how you're counting, but by my calculations Wisconsin: 53.7/45.4 -> 49.6/48.9 (3.8 diff) Michigan: 53.5/45.5 -> 50.5/48 (2.7 diff) PA: 52.0/47.3 -> 48.1/50.7 (3.7 diff), though will shift quite a lot as you mention So even then it's still closer to 3-4 point, not 4-8 as you mention.
  - divbzero 6y ago
    I think we all need to be mindful of forming conclusions too soon based on incomplete data. It’s easy to get fixated on our first impressions even as new data comes in.
  - silvester23 6y ago
    I think GP is measuring the difference in the total margin while you are measuring the difference in each candidates percentage. I.e. in Wisconsin, the forecasted margin was 8.3 points and looks to be closer to 0.7 points which is a difference of ~8 points as GP said.

ars 6y ago

> but that's the whole point of confidence intervals. That would make sense if some parts of the country had errors one way, and others another way. i.e. random error. But if every part of the country has an error, all in the same direction, then there's something larger going on.

dragonwriter 6y ago
> That would make sense if some parts of the country had errors one way, and others another way. i.e. random error. As Silver pointed out in 2016 (and why his model then gave Trump a much better chance than the NYTimes model), historically, when election outcomes differ from polls in either direction, the polling error between states is correlated, not independent. That is, it is, to all evidence, a historical fact that poll-based predictions must account for that within election cycles, there is systematic polling error, even if the direction and magnitude of that error is not consistent between cycles.
akersten 6y ago
> i.e. random error. Did they say that their confidence interval was confidence against random error? I don't think that's a requirement of a confidence interval, I think it's certainly valid to encode e.g. the chance of "shy Trump" respondents in a confidence interval.
slg 6y ago
The results of state elections aren't independent. The problems that cause most errors are going to exist in multiple states and cause the errors to be correlated.
- thaumasiotes 6y ago
  But something that causes the results everywhere to be off in the same direction isn't the kind of error you say "well, that's why you don't make point predictions" about. It's a factor that has a significant, predictable influence on the phenomenon you're studying; if it's not in your model, that means you screwed up when making the model. The whole point of the model is to account explicitly for such factors.
esrauch 6y ago
I don't think it's expecyed the errors would be random: that the polling errors are correlated and not random is something that Nate Silver talks about at length as part of the key interpretation error that others made in 2016.
Retric 6y ago
No that says nothing about the polling just your model. 538 had an interesting page where picking a state would influence the model and change what other states where predicted to do. But I can’t seem to find it now. Figuring out the dependencies between demographics is how you come up with a useful model rather than just simulating everything independently based on raw polls numbers. Aka Michigan and Wisconsin have a lot in common where Florida is influenced by wildly different racial, demographic, and economic factors.
- ehsankia 6y ago
  https://projects.fivethirtyeight.com/trump-biden-election-map/ https://projects.fivethirtyeight.com/trump-biden-election-ma...
  - Retric 6y ago
    Thanks, that’s what I was looking for. Back to my point, click Trump for Florida (which was considered a 31% chance) and the prediction looks a lot like our current map.
  - kgwgk 6y ago
    There is something similar for the Economist model: https://www.ricardofernholz.com/election/ https://www.ricardofernholz.com/election/
SamBam 6y ago
That's not their method at all. They're very insistent on the fact that voting shifts or errors between states are generally highly correlated. That's exactly why their probability forecasts tend to be more conservative than other forecasts. If you see a whole lot of possible paths for a candidate, and assume the errors are random, then that candidate will almost certainly win. If you know they might be correlated, then you know that a huge number of paths may shift at once.
deleted 6y ago
[deleted]

ttesttom 6y ago

This leaves out the states that originally tipped the election to trump last time due to polling error of 4-7pts. Really its big polling errors in these states. The rest of the map doesn't look as bad. Popular vote also most likely off by estimated 4pts when all ballots counted, which isn't terrible. Diff on final 538 -> current results Wisc: ~8pt diff Michigan: 4-6pt diff PA: still counting but most likely 2-5pts off

ehsankia 6y ago
Not sure how you're counting, but by my calculations Wisconsin: 53.7/45.4 -> 49.6/48.9 (3.8 diff) Michigan: 53.5/45.5 -> 50.5/48 (2.7 diff) PA: 52.0/47.3 -> 48.1/50.7 (3.7 diff), though will shift quite a lot as you mention So even then it's still closer to 3-4 point, not 4-8 as you mention.
- divbzero 6y ago
  I think we all need to be mindful of forming conclusions too soon based on incomplete data. It’s easy to get fixated on our first impressions even as new data comes in.
- silvester23 6y ago
  I think GP is measuring the difference in the total margin while you are measuring the difference in each candidates percentage. I.e. in Wisconsin, the forecasted margin was 8.3 points and looks to be closer to 0.7 points which is a difference of ~8 points as GP said.