4 ms·

I disagree. Probabilities can be useful even if they're not 0% or 100%. For example weather forecasts often give the probability of rain. My plans will be diffe

I disagree. Probabilities can be useful even if they're not 0% or 100%. For example weather forecasts often give the probability of rain. My plans will be different if the probability is 20% vs 80%. My plans don't tend to vary based on who wins the election, but I imagine that that information is very useful to some people. And if your data is telling you to be 80% sure, I don't think you should fudge it and claim 100%. Report the uncertainty that you actually have.

safgasCVS 8y ago
But look at what you did - you looked at the prob and then made a decision. And that’s kinda Talebs whole spiel - quoting probabilities is moot without an associated action else its just masturbating for attention. Nate shows the probabilities and puts his hands up and says well it could be anything etc. It’s reminds one of those pundits who are bullish on a stock but won’t put their money where their mouths are. If you the viewer buy it because of their recommendations and they are right then they are happy to get the praise but if it goes wrong and you the viewer pay the price they get to say “well the market is highly uncertain etc”. If the guy won’t eat his own dog food then why should others etc
lftl 8y ago
With the weather man, I could keep track of when it rains and doesn't. If it rains significantly more or less than 80% of the time they say there's an 80% chance of rain, I can say that their model is bad. I can't think of a similar way to evaluate Silver's election forecasting model. They very clearly aren't independent probabilities, and his model changes significantly from cycle to cycle. Was his model good in 2012 when every state went to where he predicted the likely probability was? Was it bad when his model said Hillary had a 71.4% chance of winning?
- danso 8y ago
  You can’t come up with a way to compare his predictions with how the elections end up turning out? Can’t you look up the results of each election?
- JumpCrisscross 8y ago
  > I can't think of a similar way to evaluate Silver's election forecasting mode You bucket every prediction, look at the outcome, and then confirm whether the favoured outcomes in the 8th decile actually occurred 70% to 80% of the time.
- matt4077 8y ago
  They do not only predict top-line presidential results, but every race, in every state, for president, House, and Senate. Non-independence is accounted for in the model, so they have no qualms about you judging them by the calibration of their predictions, I. e. You want roughly 60% of their “60%” forecasts to be right, and 40% to be wrong. If all of the races they predict 60/40 go to the more likely candidate, they themselves consider this a failure: https://fivethirtyeight.com/features/how-fivethirtyeights-2018-midterm-forecasts-did/ https://fivethirtyeight.com/features/how-fivethirtyeights-20...
tnzn 8y ago
But do weather forecast always end with "but this could also be completely random" to avoid being accountable ?

safgasCVS 8y ago

But look at what you did - you looked at the prob and then made a decision. And that’s kinda Talebs whole spiel - quoting probabilities is moot without an associated action else its just masturbating for attention. Nate shows the probabilities and puts his hands up and says well it could be anything etc. It’s reminds one of those pundits who are bullish on a stock but won’t put their money where their mouths are. If you the viewer buy it because of their recommendations and they are right then they are happy to get the praise but if it goes wrong and you the viewer pay the price they get to say “well the market is highly uncertain etc”. If the guy won’t eat his own dog food then why should others etc

lftl 8y ago

With the weather man, I could keep track of when it rains and doesn't. If it rains significantly more or less than 80% of the time they say there's an 80% chance of rain, I can say that their model is bad. I can't think of a similar way to evaluate Silver's election forecasting model. They very clearly aren't independent probabilities, and his model changes significantly from cycle to cycle. Was his model good in 2012 when every state went to where he predicted the likely probability was? Was it bad when his model said Hillary had a 71.4% chance of winning?

danso 8y ago
You can’t come up with a way to compare his predictions with how the elections end up turning out? Can’t you look up the results of each election?
JumpCrisscross 8y ago
> I can't think of a similar way to evaluate Silver's election forecasting mode You bucket every prediction, look at the outcome, and then confirm whether the favoured outcomes in the 8th decile actually occurred 70% to 80% of the time.
matt4077 8y ago
They do not only predict top-line presidential results, but every race, in every state, for president, House, and Senate. Non-independence is accounted for in the model, so they have no qualms about you judging them by the calibration of their predictions, I. e. You want roughly 60% of their “60%” forecasts to be right, and 40% to be wrong. If all of the races they predict 60/40 go to the more likely candidate, they themselves consider this a failure: https://fivethirtyeight.com/features/how-fivethirtyeights-2018-midterm-forecasts-did/ https://fivethirtyeight.com/features/how-fivethirtyeights-20...

tnzn 8y ago

But do weather forecast always end with "but this could also be completely random" to avoid being accountable ?