5 ms·
Ahh, that's even worse for Nate Silver then, if the "polls-plus" and the "polls only" predictions were not assuming the election happened today (which it seems
by Tomminn 6y ago
Ahh, that's even worse for Nate Silver then, if the "polls-plus" and the "polls only" predictions were not assuming the election happened today (which it seems like they weren't, even in 2016 [1]), then Taleb's criticism is all the more accurate.
Whatever Nate Silver was doing to incorporate future uncertainty in 2016 was clearly inadequate for all the reasons that Taleb pointed out (you could reliably make money betting on Silver's prediction odds to tend toward their time-smoothed average, implying that their time-smoothed average was a better representation of the probability than the probability itself).
[1] https://web.archive.org/web/20160930201714/http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/ https://web.archive.org/web/20160930201714/http://fivethirty...
- dragonwriter 6y agoTaleb's criticism is inaccurate in any case; it would be a valid criticism of the model if, in a large number of election cycles, the 2016 shape was typical. Because what you'd expect a forecast model to do typically is to get gradually more certain over time, so if it consistently behaved the way the 2016 forecast did, that would be a problem. But, you'd also expect, in a large number of cycles, to have some where you get patterns where you do start getting runa of information that contradict the impact of prior information, and where you do have the kind of reverses seen in the 2016 cycle. And, while we don't have enough Presidential cycles to make firm descriptions of what a “typical” cycle would look like with Silver's model, we do know that both 2012 and 2020 look a lot more like you'd expect a typical cycle in a forecast model to look.