4 ms·
There is an easy, quantifiable metric that uses this idea: the Brier score. Essentially, if you make 100 predictions, the Brier score will measure how well you
by trendia 9y ago
There is an easy, quantifiable metric that uses this idea: the Brier score.
Essentially, if you make 100 predictions, the Brier score will measure how well you are able to make precise predictions. If you always estimate the odds that X will win to be 50/50, you won't do nearly as well as someone who correctly predicts the outcome each time.
https://en.m.wikipedia.org/wiki/Brier_score https://en.m.wikipedia.org/wiki/Brier_score
- morelandjs 9y agoI didn't know about this, thanks. In the paper (long read I know), I use a modified quantile-quantile test to demonstrate that my predicted uncertainties are accurate. https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot