3 ms·

"if the election were to happen today, what is the probability of each candidate winning?" This was a correct characterization of Nate Silver's model at the ti

"if the election were to happen today, what is the probability of each candidate winning?" This was a correct characterization of Nate Silver's model at the time Nassim Taleb made his critique. Since then, Silver has changed what the probabilities in his prediction model mean. This change was (briefly) discussed previously on HN [1]. For the record, I think Taleb was an asshole, but an asshole with an entirely valid point in this debate. And I think Silver reluctantly and quietly took it on board, and then went on to use bolt-on fudge factors so that the time-dependent predictions of his model roughly look like the time-dependent predictions generated by the rigorously defined approach (option price theory) that Taleb was calling for. [1] https://news.ycombinator.com/item?id=24164814 https://news.ycombinator.com/item?id=24164814

ashtonkem 6y ago
This was a correct description of one of 538’s models, the nowcast. It was never the one they recommended, and they did not bring it back later because it tended to confuse people. They actually made it very clear on their podcasts that the nowcast was a mistake.
- Tomminn 6y ago
  Ahh, that's even worse for Nate Silver then, if the "polls-plus" and the "polls only" predictions were not assuming the election happened today (which it seems like they weren't, even in 2016 [1]), then Taleb's criticism is all the more accurate. Whatever Nate Silver was doing to incorporate future uncertainty in 2016 was clearly inadequate for all the reasons that Taleb pointed out (you could reliably make money betting on Silver's prediction odds to tend toward their time-smoothed average, implying that their time-smoothed average was a better representation of the probability than the probability itself). [1] https://web.archive.org/web/20160930201714/http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/ https://web.archive.org/web/20160930201714/http://fivethirty...
  - dragonwriter 6y ago
    Taleb's criticism is inaccurate in any case; it would be a valid criticism of the model if, in a large number of election cycles, the 2016 shape was typical. Because what you'd expect a forecast model to do typically is to get gradually more certain over time, so if it consistently behaved the way the 2016 forecast did, that would be a problem. But, you'd also expect, in a large number of cycles, to have some where you get patterns where you do start getting runa of information that contradict the impact of prior information, and where you do have the kind of reverses seen in the 2016 cycle. And, while we don't have enough Presidential cycles to make firm descriptions of what a “typical” cycle would look like with Silver's model, we do know that both 2012 and 2020 look a lot more like you'd expect a typical cycle in a forecast model to look.
raverbashing 6y ago
That's an interest take on it I'd say the way of presenting results as a "sum to 100%" fails to present uncertainty in an intuitive way
moultano 6y ago
It was not. You are mistaken about the history.

ashtonkem 6y ago

This was a correct description of one of 538’s models, the nowcast. It was never the one they recommended, and they did not bring it back later because it tended to confuse people. They actually made it very clear on their podcasts that the nowcast was a mistake.

Tomminn 6y ago
Ahh, that's even worse for Nate Silver then, if the "polls-plus" and the "polls only" predictions were not assuming the election happened today (which it seems like they weren't, even in 2016 [1]), then Taleb's criticism is all the more accurate. Whatever Nate Silver was doing to incorporate future uncertainty in 2016 was clearly inadequate for all the reasons that Taleb pointed out (you could reliably make money betting on Silver's prediction odds to tend toward their time-smoothed average, implying that their time-smoothed average was a better representation of the probability than the probability itself). [1] https://web.archive.org/web/20160930201714/http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/ https://web.archive.org/web/20160930201714/http://fivethirty...
- dragonwriter 6y ago
  Taleb's criticism is inaccurate in any case; it would be a valid criticism of the model if, in a large number of election cycles, the 2016 shape was typical. Because what you'd expect a forecast model to do typically is to get gradually more certain over time, so if it consistently behaved the way the 2016 forecast did, that would be a problem. But, you'd also expect, in a large number of cycles, to have some where you get patterns where you do start getting runa of information that contradict the impact of prior information, and where you do have the kind of reverses seen in the 2016 cycle. And, while we don't have enough Presidential cycles to make firm descriptions of what a “typical” cycle would look like with Silver's model, we do know that both 2012 and 2020 look a lot more like you'd expect a typical cycle in a forecast model to look.

raverbashing 6y ago

That's an interest take on it I'd say the way of presenting results as a "sum to 100%" fails to present uncertainty in an intuitive way

moultano 6y ago

It was not. You are mistaken about the history.