> Practically what this means is that they do not predict a winner or looser but instead report a likelihood Yes, this is good. It means that you can evaluate

> Practically what this means is that they do not predict a winner or looser but instead report a likelihood Yes, this is good. It means that you can evaluate their accuracy to a greater degree than % of times correct. > Further complicating the issue, these predictions are reported as point estimates They've learned from this, and now show their distribution of results very clearly. However, there's nothing mathematically fraught about reporting a prediction as a point estimate. > The problem is that models are not perfect replicas of the real world and are, as a matter of fact, always wrong in some way. ...yes? And so what? This is a fully general counterargument to all of science. > Predictions have two types of uncertainty; aleatory and epistemic. Don't say things like this with 100% certainty when your source itself says "the validity of this categorization is open to debate", and also when your source is Wikipedia. > However, as you can see, there is still a noticable variation of 2–5% of actual proportion to predictions. This is a signal of un-addressed epistemic uncertainty. It also means you cannot take one of these forecast probabilities at face value. Models aren't perfect. Also, 538 is very careful to address their epistemic uncertainty! They also try very hard to not change their models significantly after they publish them, in order to avoid letting their personal biases tinker with the results. This post goes out of it's way a to avoid mentioning that there are pretty well established ways of measuring the accuracy of predictions, instead using things like "look, it's not quite a line" and "look, the line went up and down before the election". If you're interested in actually reading about this from a mathematical viewpoint, read Taleb's paper, which has some actually interesting thoughts on why 538's algorithms are too eager, or read Madeka's paper, which includes comparisons of 538 to other people trying to make predictions, and finds that actually, they were better than most other news sources. https://arxiv.org/pdf/1703.06351.pdf https://arxiv.org/pdf/1703.06351.pdf https://arxiv.org/pdf/1704.02664.pdf https://arxiv.org/pdf/1704.02664.pdf