Every event is a one-off event. In particular the Trump election was the most one-off event in recent memory :) Kidding aside, your observation is correct, you

Every event is a one-off event. In particular the Trump election was the most one-off event in recent memory :) Kidding aside, your observation is correct, you need to perform repeated predictions. 538 does just that. They keep predicting a whole lot of outcomes, so you can check their track record. They even have a challenge for the audience, where you can record your own predictions and compare them with their corresponding predictions (see for example [1] for NFL games). The scoring of the predictions uses the Brier score [2], which is just a version of R-squared for classification problems. In the case of the November 2016 election, other prediction sites were giving the Trump team a 1-2% chance, 538 was giving them a 20% change. Considering that, 538 comes out as the clear winner. Which brings us to what I consider to be the best measure of prediction accuracy (my own invention, it doesn't have a scientific name yet): you can compare two sequences of predictions (let's say yours agains 538) by simulating bets at some mid-odds level. For example if I say Trump's probability of winning 2020 is 45% and you say it's 35%, then I'm willing to give you 2-to-3 odds, while you're willing to take that. We then see who stays ahead most of the time in this betting simulation. For what is worth, here's a recent survey of different scoring types for predictions: "Assessing the performance of prediction models: a framework for traditional and novel measures" [3] [1] https://projects.fivethirtyeight.com/nfl-predictions-game/ https://projects.fivethirtyeight.com/nfl-predictions-game/ [2] https://en.wikipedia.org/wiki/Brier_score https://en.wikipedia.org/wiki/Brier_score [3] https://www.ncbi.nlm.nih.gov/pubmed/20010215 https://www.ncbi.nlm.nih.gov/pubmed/20010215