3 ms·

More like saying, "We said there was a 17% chance that it could go the other way, so no matter what happens, we aren't wrong."

• Symmetry 7y ago
  That's why it's important to do what 538 did here. Look at all the different places where they made 5 in 6 predictions ( and there are a lot of different races) and make sure that 5 in 6 of them happened.
• b_tterc_p 7y ago
  This is disingenuous. A prediction is a prediction. There is a world of difference between 51% certain and 99% certain. People excited that their data science models get good predictions on their test set without considering, say, the RoC curve are setting themselves up for failure. If a model says 75% chance of X rather than Y, that doesn’t mean you should expect X. Depending on context, you might be better off flagging it as unclear or even safer to conclude a possible Y. Testing calibration of many models across many different trials as a test of data scientist competence is a little strange, but if you want to answer the question of how valuable the predicted likelihood’s are, it’s a decent measure.