3 ms·

Bringing in betting markets has nothing to do with the point, but you are right on one thing: if you had bet along with Nate’s odds in a simulation you’d lose m

Bringing in betting markets has nothing to do with the point, but you are right on one thing: if you had bet along with Nate’s odds in a simulation you’d lose money in a Monte Carlo simulator based on current results. The end story is the polls weren’t a reliable measure of the vote. Their predictive ability was low, and even playing by the rules of Nate’s model you have to admit he was off by some 30% at minimum. Nate’s model has the average PA poll at Biden +6 (!!) where the final result looks to be under 1 (and could go either way). It’s simply impossible to argue that was an accurate model.

awb 6y ago
> Nate’s model has the average PA poll at Biden +6 (!!) where the final result looks to be under 1 (and could go either way). It’s simply impossible to argue that was an accurate model. 538's PA indicator ended at Biden +4.7%. But the overall 538 model doesn't care if a candidate wins PA by +0.7% or +4.7%. The 538 model is designed to do 1 thing: predict the EC winner. It uses state-wide predictions as indicators, but you can't judge a model based on the performance of a single low-level indicator, you judge it by it's final prediction, which is Biden 90% / Trump 10% to win the EC. But yes, the lower you go in the model the higher variance you'll see: Level 1 -> EC Winner (538's focus -- could end up 100% correct) Level 2 -> State Winners (low variance -- could end up 90-95% correct) Level 3 -> State Polling Averages (moderate variance & could be 75-90% within margin of error) Level 4 -> Individual Polls (high variance & could be 60-80% within margin of error) If you click on any individual state (like PA: https://projects.fivethirtyeight.com/2020-election-forecast/pennsylvania/ https://projects.fivethirtyeight.com/2020-election-forecast/...), you can see the state-wide vote projections. Biden was projected to earn between 49 and 55% of PA's vote with Trump expected to get between 45 and 50%. The final outcome looks certain to fall within those expected ranges. Again, the margin really doesn't matter, just that the idea of Trump getting above 50% and Biden getting below 50% seemed pretty difficult, hence Trump's low (but not impossible) 16% chance of carrying the state. I like the way their "Winding Path to Victory" chart (https://projects.fivethirtyeight.com/2020-election-forecast https://projects.fivethirtyeight.com/2020-election-forecast) explains their model. It's basically saying: "Trump's path to 270 likely goes through PA, NE-2, AZ, FL, NC, ME-2, GA, OH, IA & TX, where he has a fighting chance of winning any of those, but a very low chance at winning all of them." On the flip side, "Biden's path to 270 goes through Wisconsin, Michigan, Nevada and Pennsylvania, where he's likely to win all of them". That seems like a pretty reasonable EC prediction to me. If Silver was promoting the model as being able to accurately predict state vote percentages, I'd agree that it's underwhelming. But when you average all of that variability and uncertainty, you can get a pretty reasonable EC winner prediction IMO, at least better than others I've seen that had Biden closer to 95 or 98% to win the EC, or betting markets that had him as low as 60%.
- nwienert 6y ago
  If every single range it’s at at or below the lower bound for one candidate and at or just past the upper bound for the other, systematically, that’s the definition of a flawed model. You seem to breeze past every point I make, I suppose there’s no educating the unwilling so I’ll leave this thread. The bell curve was systemically off, the polls were systemically off. They actually missed the bounds in many races entirely. The betting markets were much more accurate. I’ll leave this here: https://mobile.twitter.com/NateSilver538/status/1322877824089378817?s=19 https://mobile.twitter.com/NateSilver538/status/132287782408... About 4pts off on average across them all in the same direction, all past the lower bound.
  - stopachka 6y ago
    I really enjoyed reading your arguments. Thank you for going deep!
  - awb 6y ago
    > If every single range it’s at at or below the lower bound for one candidate and at or just past the upper bound for the other, systematically, that’s the definition of a flawed model. Yes, I agree that individual low-level polls for 2 elections in a row have underrepresented Trump's actual support. But, 538's model addresses exactly your point: low-level indicators like state polling can potentially be systematically flawed and biased. By simulating those unreliable low-level indicators through 40k scenarios, the scenario where Biden systematically underperforms biased pre-election polls in key swing states but Biden still wins (as we're seeing) ends up being a very reasonable outcome and the large reason why Biden was favored overall. Trump needed to outperform the polls in 7 states to win, and that just didn't happen. And the polls weren't off dramatically in every single race. Here are some of 538's last predictions in key states: AZ: 50.7% Biden / 48.1% Trump NE-2: 51% Biden / 47.8% Trump GA: 50.1% Biden / 49.2% Trump PA: 52% Biden / 47.3% Trump NC: 50.5% Biden / 48.8% Trump Each poll has a margin of error, usually +/-2.5% or more. So, 2-3% swings on Election day are completely normal. NC was off by a tiny margin and enough to flip the state to Trump, while the others were also off by tiny margins, but not enough to flip the state. And here's how much polling has traditionally been off: https://en.wikipedia.org/wiki/Historical_polling_for_United_States_presidential_elections https://en.wikipedia.org/wiki/Historical_polling_for_United_... Sizable polling swings are not uncommon and historically have gone in both directions, but the model was more concerned about just how narrow Trump's path to 270 was, more so than a routine or even historical polling error. > The betting markets were much more accurate. Everyone who bet on Trump at a 30/40% chance to win is about to lose their bet. I definitely wouldn't have taken those odds for Trump to run the table in 7 straight must-win swing states all polling as a coin-flip and within the margin of error. In a completely random scenario Trump would have only had a 14% chance of pulling that off.

awb 6y ago

> Nate’s model has the average PA poll at Biden +6 (!!) where the final result looks to be under 1 (and could go either way). It’s simply impossible to argue that was an accurate model. 538's PA indicator ended at Biden +4.7%. But the overall 538 model doesn't care if a candidate wins PA by +0.7% or +4.7%. The 538 model is designed to do 1 thing: predict the EC winner. It uses state-wide predictions as indicators, but you can't judge a model based on the performance of a single low-level indicator, you judge it by it's final prediction, which is Biden 90% / Trump 10% to win the EC. But yes, the lower you go in the model the higher variance you'll see: Level 1 -> EC Winner (538's focus -- could end up 100% correct) Level 2 -> State Winners (low variance -- could end up 90-95% correct) Level 3 -> State Polling Averages (moderate variance & could be 75-90% within margin of error) Level 4 -> Individual Polls (high variance & could be 60-80% within margin of error) If you click on any individual state (like PA: https://projects.fivethirtyeight.com/2020-election-forecast/pennsylvania/ https://projects.fivethirtyeight.com/2020-election-forecast/...), you can see the state-wide vote projections. Biden was projected to earn between 49 and 55% of PA's vote with Trump expected to get between 45 and 50%. The final outcome looks certain to fall within those expected ranges. Again, the margin really doesn't matter, just that the idea of Trump getting above 50% and Biden getting below 50% seemed pretty difficult, hence Trump's low (but not impossible) 16% chance of carrying the state. I like the way their "Winding Path to Victory" chart (https://projects.fivethirtyeight.com/2020-election-forecast https://projects.fivethirtyeight.com/2020-election-forecast) explains their model. It's basically saying: "Trump's path to 270 likely goes through PA, NE-2, AZ, FL, NC, ME-2, GA, OH, IA & TX, where he has a fighting chance of winning any of those, but a very low chance at winning all of them." On the flip side, "Biden's path to 270 goes through Wisconsin, Michigan, Nevada and Pennsylvania, where he's likely to win all of them". That seems like a pretty reasonable EC prediction to me. If Silver was promoting the model as being able to accurately predict state vote percentages, I'd agree that it's underwhelming. But when you average all of that variability and uncertainty, you can get a pretty reasonable EC winner prediction IMO, at least better than others I've seen that had Biden closer to 95 or 98% to win the EC, or betting markets that had him as low as 60%.

nwienert 6y ago
If every single range it’s at at or below the lower bound for one candidate and at or just past the upper bound for the other, systematically, that’s the definition of a flawed model. You seem to breeze past every point I make, I suppose there’s no educating the unwilling so I’ll leave this thread. The bell curve was systemically off, the polls were systemically off. They actually missed the bounds in many races entirely. The betting markets were much more accurate. I’ll leave this here: https://mobile.twitter.com/NateSilver538/status/1322877824089378817?s=19 https://mobile.twitter.com/NateSilver538/status/132287782408... About 4pts off on average across them all in the same direction, all past the lower bound.
- stopachka 6y ago
  I really enjoyed reading your arguments. Thank you for going deep!
- awb 6y ago
  > If every single range it’s at at or below the lower bound for one candidate and at or just past the upper bound for the other, systematically, that’s the definition of a flawed model. Yes, I agree that individual low-level polls for 2 elections in a row have underrepresented Trump's actual support. But, 538's model addresses exactly your point: low-level indicators like state polling can potentially be systematically flawed and biased. By simulating those unreliable low-level indicators through 40k scenarios, the scenario where Biden systematically underperforms biased pre-election polls in key swing states but Biden still wins (as we're seeing) ends up being a very reasonable outcome and the large reason why Biden was favored overall. Trump needed to outperform the polls in 7 states to win, and that just didn't happen. And the polls weren't off dramatically in every single race. Here are some of 538's last predictions in key states: AZ: 50.7% Biden / 48.1% Trump NE-2: 51% Biden / 47.8% Trump GA: 50.1% Biden / 49.2% Trump PA: 52% Biden / 47.3% Trump NC: 50.5% Biden / 48.8% Trump Each poll has a margin of error, usually +/-2.5% or more. So, 2-3% swings on Election day are completely normal. NC was off by a tiny margin and enough to flip the state to Trump, while the others were also off by tiny margins, but not enough to flip the state. And here's how much polling has traditionally been off: https://en.wikipedia.org/wiki/Historical_polling_for_United_States_presidential_elections https://en.wikipedia.org/wiki/Historical_polling_for_United_... Sizable polling swings are not uncommon and historically have gone in both directions, but the model was more concerned about just how narrow Trump's path to 270 was, more so than a routine or even historical polling error. > The betting markets were much more accurate. Everyone who bet on Trump at a 30/40% chance to win is about to lose their bet. I definitely wouldn't have taken those odds for Trump to run the table in 7 straight must-win swing states all polling as a coin-flip and within the margin of error. In a completely random scenario Trump would have only had a 14% chance of pulling that off.