4 ms·
"So if Romney had won Florida, Silver would have been wrong, but only a little wrong." No, Silver would not have been wrong at all, or at least we'll never kno
by hsitz 10y ago
"So if Romney had won Florida, Silver would have been wrong, but only a little wrong."
No, Silver would not have been wrong at all, or at least we'll never know. He doesn't predict a winner; he gives probabilities of victory. It would have been perfectly consistent with Obama having a 50.3% chance of winning (or 63.5% or 32.8% or 99.9% or whatever probability you want to assign) and then for Romney to have won. Even after the election was over, a Florida victory for Romney would not have meant that Silver's pre-election probability assignment of 50.3% to Obama was wrong. That's not how probabilities work.
Think of sporting events. A huge underdog with only, say, a 2% chance of winning may beat the favorite. This does not mean that it had better than a 2% chance of winning before the event. Depending on the reason for the underdog's victory, it may not even mean that the underdog has better than a 2% chance of winning if they play again. One in a million events happen, and it doesn't mean that their chances of happening were greater than one in a million.
- basch 10y agoread the last quoted sentence
- hsitz 10y agoNot sure which sentence you're referring to. And I'm not saying there's no way to tell which probability assignments are better. I'm just saying that the mere fact that an event assigned a lower probability happens (e.g., if Romney would have beaten Obama in Florida) does not mean the assignments were wrong. This is consistent with Silver's own explanations of what he's doing, and (I'm pretty sure ) he is careful to say he doesn't make "predictions". (Actually, not sure, it may be merely that he often clarifies that a different result than predicted does not make the probability assignment that prompted the prediction "wrong". The prediction and the probability analysis it's based on are different things.) I'm not saying that the Brier Score can't differentiate between better and worse probability assignments, but I am saying it's confusing the issue if it calls probability assignments "right" or "wrong". They may be "better" or "worse" than each other, but not "right" or "wrong". This may be using language in more technical manner, but it seems to me (and to Silver) to be an important distinction.
- basch 10y ago"Also note that Wang & Ferguson got a better Brier score than Silver despite getting Florida (barely) wrong while Silver got Florida right. The Atlantic Wire gave Wang & Ferguson only a “Silver Star” for this reason, but our more detailed analysis shows that Wang & Ferguson probably should have gotten a “Gold Star.”" I think being concerned about this authors usage of the words "right and wrong" misses the point. The media will say "you were wrong" if you were on the wrong side of a 51/49% prediction, even if you shouldn't be penalized for it. Wang & Ferguson got dinged by The Atlantic Wire for being wrong, when they shouldnt have. The last quote in my post, supports your point. The person who is least wrong is the person with the lowest aggregate Brier score. The author was using right and wrong, because the articles they were referencing used those terms. The quote was written with the colloquial language of what it addressed.
- hsitz 10y agoWe're not too far off. Main thng in original quote that bothered me was "So if Romney had won Florida, Silver would have been wrong, but ONLY A LITTLE WRONG." Predictions of victory or loss are either right or wrong. There are not degrees of rightness or wrongness with the prediction; it's either right or wrong. The probability analysis that the prediction is based on does admit of degrees; one probability assignment may be better or worse than another. But saying a prediction is "only a little wrong" implicitly refers to the probability assignment it was based on; it confuses the concepts of "prediction" and "probability". The probability assignment was not necessarily "wrong" to assign a higher probability to the event that didn't occur. It may have been a bad probability assignment, but discerning that would involve considering factors other than the outcome of the event.