5 ms·
Yeah, "extremely possible" is perhaps a poor choice of words, and could be interpreted uncharitably as a hedge against any outcome, but I think there's an obvio
by tshaddox 8y ago
Yeah, "extremely possible" is perhaps a poor choice of words, and could be interpreted uncharitably as a hedge against any outcome, but I think there's an obvious charitable interpretation too: that things that have a 20% chance of occurring turn out to occur around 20% of the time. No one would be blown away by surprise if I proclaimed "SIX!" then rolled a six-sided die and got a six, but that's even less likely than 20%.
So it's reasonable to say that, if 538 claims that an event has a 20% probability of happening, and that event happens, no one should be astonished. Of course, such events ought to happen about 20% of the time, if 538's models are good. And that evaluation is precisely what 538 published a few days ago: https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/
- Bjartr 8y agoReminds me of the old essay on "Words of Estimative Probability" from the CIA https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/sherman-kent-and-the-board-of-national-estimates-collected-essays/6words.html https://www.cia.gov/library/center-for-the-study-of-intellig...
- ohashi 8y agoThat's a great table they have with associated words and numerical probability associated with them.
- tcbawo 8y agoLoads of people criticized Silver for not having "predicted" Trump's electoral victory. I imagine Silver's language was trying to hand-hold people to an understanding that even 20% probability events happen 1 out of 5 times. With so many people trying to twist and spin every headline, prediction, number, I appreciate Silver's attempt to make a reasonable guess with transparent methodologies.
- jandrese 8y agoThe same people were criticizing him a day before because everybody else had Hillary at a 99% chance to win. Nate tweets that Trump is one standard polling error away from winning and people scoffed.
- Someone1234 8y agoIt is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but people infer from the final datapoint to the conclusion later that same day about 538's accuracy. What I am saying is: 538 needs to keep a historical record of their predictions.
- wtallis 8y ago538 published this last week: https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ https://fivethirtyeight.com/features/when-we-say-70-percent-...
- dandelany 8y agoMaybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It’s still visible here: https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/#change https://projects.fivethirtyeight.com/2018-midterm-election-f...
- dragonwriter 8y ago> Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It has not changed. They did this the whole way back when it was an independent blog before the NY Times (later ESPN then ABC News) picked them up.