3 ms·

Um, we can tell after the election is over? I mean, that's generally what a forecast is tested by - the outcome versus the prediction. Every scenario is differe

Um, we can tell after the election is over? I mean, that's generally what a forecast is tested by - the outcome versus the prediction. Every scenario is different, so relying on one model is only good for one test I think. Not saying building models is useless, I just think the analytical part is flawed in general because it still relies on humans to respond and humans are...difficult... I know a guy who correctly predicted the Obama/Romney popular vote outcome within 1% of each's actual take IIRC. As of now, he's got Clinton at 47% and Trump 34%, based on nothing more than personal outlook, hunches, and mulling over various polling data (he had 47% Clinton to 41% Trump just before the Conventions). Point being it might be just as likely an irrational human can point out human behavior (en masse, e.g. voting) on par with a statistical model, given some particular circumstances.

1337biz 10y ago
Hey, don't tell the kids Santa Clause is not real. We need to hold up the believe that there is some magical wizard who is the only one knowing how to do polling the correct way.
matt4077 10y ago
The problem with that it only tells us how good they were on election day. It'd be great to somehow evaluate their earlier predictions, which is probably only possible by looking at an aggregate of election. (or maybe, since there are predictions and results for every state and every senate race, it's actually possible to use those for an evaluation. Note that it's not as easy as scoring a "won by trump" in favor of a forecaster who was giving trump a 60% chance. You need a measure that expects your 60% predictions to be wrong 40% of the time, like the Brier score)
lintiness 10y ago
it's a binary event, so differences in degree are going to be difficult to measure (especially in outcome). poker has an interesting add: don't measure your play by outcome; get the process right and results will follow. if you focus on outcome, you're likely to believe you played poorly when in reality you were just unlucky (something unlikely happened).

1337biz 10y ago

Hey, don't tell the kids Santa Clause is not real. We need to hold up the believe that there is some magical wizard who is the only one knowing how to do polling the correct way.

matt4077 10y ago

The problem with that it only tells us how good they were on election day. It'd be great to somehow evaluate their earlier predictions, which is probably only possible by looking at an aggregate of election. (or maybe, since there are predictions and results for every state and every senate race, it's actually possible to use those for an evaluation. Note that it's not as easy as scoring a "won by trump" in favor of a forecaster who was giving trump a 60% chance. You need a measure that expects your 60% predictions to be wrong 40% of the time, like the Brier score)

lintiness 10y ago

it's a binary event, so differences in degree are going to be difficult to measure (especially in outcome). poker has an interesting add: don't measure your play by outcome; get the process right and results will follow. if you focus on outcome, you're likely to believe you played poorly when in reality you were just unlucky (something unlikely happened).