5 ms·

Why Nate Silver, Sam Wang and Everyone Else Were Wrong

8 points by digital55 10y ago

ffjffsfr 10y ago
Everyone claiming that polls or forecasters were wrong should really do their homework and think what 'forecasting' is all about. It is not about predicting future with 100% certainty. It is about calculating odds of outcomes. You can calculate odds of Trump winning and put them at 40%, you can put them at 1% or 90%. In neither of cases you are wrong or right when he wins. Odds of winning are not 'verified' in any way. If you say that there is 50% chance of coin flipping on tails you are not wrong when it flips on head.
- deleted 10y ago
  [deleted]
- gph 10y ago
  Perhaps, but I still think the fact that so many polls had Clinton's chances of winning well above 90% is indicative of some incorrect assumptions within their models. Especially considering how they were off in nearly every single swing state. Most forecasters had Clinton winning Wisc/Michigan at over 90%, and considerable advantages in most the other battleground states. It would be like if I gave you a quarter and said it has a 75% chance of it coming up heads (ignore physics), then you flip it five times and it comes up tails each time. Sure it could be a run of coincidence, but I'd also put serious consideration into my statistical model.
  - basch 10y ago
    the polls were not expecting a globalist establishment incumbent vs a nationalist populist outsider. the polls treated him like GOP candidate, when he was more of a parasite to the party, using its skin to get on every ballet as a "top two" candidate. at least Nate Silver's part two to this piece will include "i was less confident than most others" https://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ https://fivethirtyeight.com/features/how-i-acted-like-a-pund... Silver really took Trump's ascension to the GOP throne as a lesson. http://www.politico.com/story/2016/11/nate-silver-huffington-post-polls-twitter-230815 http://www.politico.com/story/2016/11/nate-silver-huffington...
totalZero 10y ago
Didn't Nate Silver acknowledge that this was a tough election for his methods because of the highly unusual nature of Trump's campaign, among other things? EDIT: found some indication of this from May of this year: http://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ http://fivethirtyeight.com/features/how-i-acted-like-a-pundi...
- zzalpha 10y ago
  And he was getting flack for giving Hillary only 2-1 odds. And he also indicated there was error bars that work in both candidate's favour. IMO, if anyone was at least sorta close, it was FiveThirtyEight.
  - Someone 10y ago
    You can't claim they were wrong at all. If you throw a die and I say there's a 67% chance you wil throw a three or higher, and you throw a one or a two, you cannot claim I am wrong. It's true that you cannot claim I was wrong if I said that probability was .00000000001%, either, but it sure feels different, and statistical tests will also bear that out.
- simonbarker87 10y ago
  I've been listening to the 538 daily podcast for the last couple of weeks (as a Brit it's been a great way to get an insight on how the US system works rather than just the politics on the surface) and, when on the show, Nate would not be drawn on saying who he thought was going to win. He really seemed to be going out of his way to say that the model is based on polls and those polls can be off, the model is only as good as the information that's put into it. The model had a pretty low Hilary chance a few days ago and then it ticked up at the end to over 70% for her but Jody Avirgan and Harry Enten were warning about pollsters' tendency to herd (I think that's what they called it) at the end and so the final polls can be less reliable due to this. I'm looking forward to their wash up podcast and then I'm checking out of politics, between Brexit and this it's been a bumpy year.
  - 10y ago
bhauer 10y ago
Actually, the LA Times made a fairly big deal about forecasting correctly. https://twitter.com/latimes/status/796360500091252736 https://twitter.com/latimes/status/796360500091252736
- deleted 10y ago
  [deleted]
- maxerickson 10y ago
  They predicted Clinton would win the electoral college.

8 points by digital55 10y ago

Everyone claiming that polls or forecasters were wrong should really do their homework and think what 'forecasting' is all about. It is not about predicting future with 100% certainty. It is about calculating odds of outcomes. You can calculate odds of Trump winning and put them at 40%, you can put them at 1% or 90%. In neither of cases you are wrong or right when he wins. Odds of winning are not 'verified' in any way. If you say that there is 50% chance of coin flipping on tails you are not wrong when it flips on head.

deleted 10y ago
[deleted]
gph 10y ago
Perhaps, but I still think the fact that so many polls had Clinton's chances of winning well above 90% is indicative of some incorrect assumptions within their models. Especially considering how they were off in nearly every single swing state. Most forecasters had Clinton winning Wisc/Michigan at over 90%, and considerable advantages in most the other battleground states. It would be like if I gave you a quarter and said it has a 75% chance of it coming up heads (ignore physics), then you flip it five times and it comes up tails each time. Sure it could be a run of coincidence, but I'd also put serious consideration into my statistical model.
- basch 10y ago
  the polls were not expecting a globalist establishment incumbent vs a nationalist populist outsider. the polls treated him like GOP candidate, when he was more of a parasite to the party, using its skin to get on every ballet as a "top two" candidate. at least Nate Silver's part two to this piece will include "i was less confident than most others" https://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ https://fivethirtyeight.com/features/how-i-acted-like-a-pund... Silver really took Trump's ascension to the GOP throne as a lesson. http://www.politico.com/story/2016/11/nate-silver-huffington-post-polls-twitter-230815 http://www.politico.com/story/2016/11/nate-silver-huffington...

totalZero 10y ago

Didn't Nate Silver acknowledge that this was a tough election for his methods because of the highly unusual nature of Trump's campaign, among other things? EDIT: found some indication of this from May of this year: http://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ http://fivethirtyeight.com/features/how-i-acted-like-a-pundi...

zzalpha 10y ago
And he was getting flack for giving Hillary only 2-1 odds. And he also indicated there was error bars that work in both candidate's favour. IMO, if anyone was at least sorta close, it was FiveThirtyEight.
- Someone 10y ago
  You can't claim they were wrong at all. If you throw a die and I say there's a 67% chance you wil throw a three or higher, and you throw a one or a two, you cannot claim I am wrong. It's true that you cannot claim I was wrong if I said that probability was .00000000001%, either, but it sure feels different, and statistical tests will also bear that out.
simonbarker87 10y ago
I've been listening to the 538 daily podcast for the last couple of weeks (as a Brit it's been a great way to get an insight on how the US system works rather than just the politics on the surface) and, when on the show, Nate would not be drawn on saying who he thought was going to win. He really seemed to be going out of his way to say that the model is based on polls and those polls can be off, the model is only as good as the information that's put into it. The model had a pretty low Hilary chance a few days ago and then it ticked up at the end to over 70% for her but Jody Avirgan and Harry Enten were warning about pollsters' tendency to herd (I think that's what they called it) at the end and so the final polls can be less reliable due to this. I'm looking forward to their wash up podcast and then I'm checking out of politics, between Brexit and this it's been a bumpy year.
- 10y ago

bhauer 10y ago

Actually, the LA Times made a fairly big deal about forecasting correctly. https://twitter.com/latimes/status/796360500091252736 https://twitter.com/latimes/status/796360500091252736

deleted 10y ago
[deleted]
maxerickson 10y ago
They predicted Clinton would win the electoral college.