3 ms·

They did not, he has not, and one should not any more than one always should. 538's forecast was easily the most accurate for November 2016, by a decent margin

They did not, he has not, and one should not any more than one always should. 538's forecast was easily the most accurate for November 2016, by a decent margin. The majority of critics of 538 are "sore winners" who delight in their side having won despite being given a less than 50% chance, but as Silver pointed out at the time: his chances were similar to those in Russian Roulette, and most people consider that too risky to play.

paulgb 8y ago
Right. It's like saying "Ha! You were wrong! You said there was a 83% chance I wouldn't roll a six, and I did"
- insickness 8y ago
  More like saying, "We said there was a 17% chance that it could go the other way, so no matter what happens, we aren't wrong."
  - Symmetry 8y ago
    That's why it's important to do what 538 did here. Look at all the different places where they made 5 in 6 predictions ( and there are a lot of different races) and make sure that 5 in 6 of them happened.
  - b_tterc_p 8y ago
    This is disingenuous. A prediction is a prediction. There is a world of difference between 51% certain and 99% certain. People excited that their data science models get good predictions on their test set without considering, say, the RoC curve are setting themselves up for failure. If a model says 75% chance of X rather than Y, that doesn’t mean you should expect X. Depending on context, you might be better off flagging it as unclear or even safer to conclude a possible Y. Testing calibration of many models across many different trials as a test of data scientist competence is a little strange, but if you want to answer the question of how valuable the predicted likelihood’s are, it’s a decent measure.
protomyth 8y ago
538's forecast was easily the most accurate for November 2016, by a decent margin. No, Investor’s Business Daily was the most accurate.
objektif 8y ago
If thats your understanding of making a prediction then he was accurate by your standards. You clearly did not watch the election night predictions. NYT live model was miles better than 538s. https://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ https://fivethirtyeight.com/features/how-i-acted-like-a-pund...
- novia 8y ago
  Your link is from 2016 before the presidential election but after the primaries. Nate is admitting that he underestimated Trump in the primaries and talks about what happened. This does not seem particularly relevant to the subject of election night predictions.
  - objektif 8y ago
    This is regarding his bias. If anything the link is even more relevant. He repeated the similiar mistakes.
    - pwinnski 8y ago
      His main mistake as highlighted in that article is that they didn't use statistics for the nomination process, but guesswork. For the election proper, they did use statistics. So no, they did not repeat the mistake at all.
- notheruser 8y ago
  His election night model wasn't good, but he isn't known and respected for his election night model, so who cares?

paulgb 8y ago

Right. It's like saying "Ha! You were wrong! You said there was a 83% chance I wouldn't roll a six, and I did"

insickness 8y ago
More like saying, "We said there was a 17% chance that it could go the other way, so no matter what happens, we aren't wrong."
- Symmetry 8y ago
  That's why it's important to do what 538 did here. Look at all the different places where they made 5 in 6 predictions ( and there are a lot of different races) and make sure that 5 in 6 of them happened.
- b_tterc_p 8y ago
  This is disingenuous. A prediction is a prediction. There is a world of difference between 51% certain and 99% certain. People excited that their data science models get good predictions on their test set without considering, say, the RoC curve are setting themselves up for failure. If a model says 75% chance of X rather than Y, that doesn’t mean you should expect X. Depending on context, you might be better off flagging it as unclear or even safer to conclude a possible Y. Testing calibration of many models across many different trials as a test of data scientist competence is a little strange, but if you want to answer the question of how valuable the predicted likelihood’s are, it’s a decent measure.

protomyth 8y ago

538's forecast was easily the most accurate for November 2016, by a decent margin. No, Investor’s Business Daily was the most accurate.

objektif 8y ago

If thats your understanding of making a prediction then he was accurate by your standards. You clearly did not watch the election night predictions. NYT live model was miles better than 538s. https://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/ https://fivethirtyeight.com/features/how-i-acted-like-a-pund...

novia 8y ago
Your link is from 2016 before the presidential election but after the primaries. Nate is admitting that he underestimated Trump in the primaries and talks about what happened. This does not seem particularly relevant to the subject of election night predictions.
- objektif 8y ago
  This is regarding his bias. If anything the link is even more relevant. He repeated the similiar mistakes.
  - pwinnski 8y ago
    His main mistake as highlighted in that article is that they didn't use statistics for the nomination process, but guesswork. For the election proper, they did use statistics. So no, they did not repeat the mistake at all.
notheruser 8y ago
His election night model wasn't good, but he isn't known and respected for his election night model, so who cares?