4 ms·

Nate Silver built an aleatory model whereby low probability high effect events (Comey reopening the investigation on eve of the election) are discounted because

Nate Silver built an aleatory model whereby low probability high effect events (Comey reopening the investigation on eve of the election) are discounted because if you add enough of these potential disruptive events the model will be 50%-50% and that has no value to anybody. Who wants to read an article that said "election forecast 50-50" every day. If a 538 model says one outcome is 75% probable, it really should come with a giant caveat saying "at current trends assuming nothing out of the ordinary occurs". Taleb's beef is that if that is the case, the 75% does not mean in reality this result is 75% likely to happen. What is a really interesting takeaway is that if in an event where the outcome you desire based appears unlikely, your job is introduce as much previously undefined or discounted uncertainty. Ideally you engineer a black swan event or at least do what you can to make it happen. This would be fun to model from a game-theory approach instead.

wtallis 8y ago
> whereby low probability high effect events (Comey reopening the investigation on eve of the election) are discounted because if you add enough of these potential disruptive events the model will be 50%-50% and that has no value to anybody. I don't think you can say this. The challenge is that it's impossible to even enumerate all the possible surprises that could drastically swing an election, and incorporating some of them into a model would require pulling numbers out of your ass for how to weight those unpredictable possibilities, and even picking which potential surprises to factor in is a similarly arbitrary decision for which there is insufficient evidence to provide guidance. But none of that means that attempting to factor in such possibilities will drive your model's predictions toward 50%; your predictions could end up almost anywhere in the unit interval depending on the value of the priors you pulled out of your ass, and your final number ends up saying more about your biases than about the state of available predictive evidence.
waterhouse 8y ago
> What is a really interesting takeaway is that if in an event where the outcome you desire based appears unlikely, your job is introduce as much previously undefined or discounted uncertainty. Ideally you engineer a black swan event or at least do what you can to make it happen. This would be fun to model from a game-theory approach instead. In chess, if you are behind, John Nunn's two recommended strategies are "grim defence" (if your opponent's advantage is not so large as to make it easy for them to force a win) and "create confusion": create complicated tactical situations and hope your opponent makes a mistake. The farther behind you are, the more appealing "create confusion" gets by comparison.
OscarCunningham 8y ago
I don't think this is a good description of Silver's model. It doesn't discount disruptive events at all. It implicitly takes them into account. Silver knows that people's opinions can be changed by events that happen before the election. This is why polls taken early on are less informative about the final result than polls taken closer to the day. Based on historical records, Silver knows how the accuracy of polls depends on the date they are taken, and he weights them accordingly. This process automatically models the possibility that an event could suddenly change people's opinions just before the election. It's taken into account in terms of the accuracy of polls.

wtallis 8y ago

> whereby low probability high effect events (Comey reopening the investigation on eve of the election) are discounted because if you add enough of these potential disruptive events the model will be 50%-50% and that has no value to anybody. I don't think you can say this. The challenge is that it's impossible to even enumerate all the possible surprises that could drastically swing an election, and incorporating some of them into a model would require pulling numbers out of your ass for how to weight those unpredictable possibilities, and even picking which potential surprises to factor in is a similarly arbitrary decision for which there is insufficient evidence to provide guidance. But none of that means that attempting to factor in such possibilities will drive your model's predictions toward 50%; your predictions could end up almost anywhere in the unit interval depending on the value of the priors you pulled out of your ass, and your final number ends up saying more about your biases than about the state of available predictive evidence.

waterhouse 8y ago

> What is a really interesting takeaway is that if in an event where the outcome you desire based appears unlikely, your job is introduce as much previously undefined or discounted uncertainty. Ideally you engineer a black swan event or at least do what you can to make it happen. This would be fun to model from a game-theory approach instead. In chess, if you are behind, John Nunn's two recommended strategies are "grim defence" (if your opponent's advantage is not so large as to make it easy for them to force a win) and "create confusion": create complicated tactical situations and hope your opponent makes a mistake. The farther behind you are, the more appealing "create confusion" gets by comparison.

OscarCunningham 8y ago

I don't think this is a good description of Silver's model. It doesn't discount disruptive events at all. It implicitly takes them into account. Silver knows that people's opinions can be changed by events that happen before the election. This is why polls taken early on are less informative about the final result than polls taken closer to the day. Based on historical records, Silver knows how the accuracy of polls depends on the date they are taken, and he weights them accordingly. This process automatically models the possibility that an event could suddenly change people's opinions just before the election. It's taken into account in terms of the accuracy of polls.