> But presumably you don't know how good they are at predicting topic D until topic D actually occurs at least once? No, that's not right. The general idea is

> But presumably you don't know how good they are at predicting topic D until topic D actually occurs at least once? No, that's not right. The general idea is that you have forecasters who make predictions on multiple topics A, B, C, and D that have all not happened at the time of the prediction. Then, by looking at the outcomes of A, B, and C you can judge the ability of the forecaster to make prediction on topics what have not yet occurred at the time of prediction. You can then become confident on their ability to predict topic D before it occurs. The argument you're making is the classic skeptical argument about the philosophical problem of induction: just because some procedure (e.g., an astronomical model of the solar system) has made successful predictions in the past, we cannot rule out the possibility that the sun will not rise tomorrow. You can see where the problem with your argument lies by noticing that it depends heavily on the definition of what a "topic" is. If we have made successful predictions in the past on (say) the tides, and I make new predictions in the future, one can always point to some feature of the new predictions that could arguably make it a different topic, e.g., maybe El Nino is this year, or maybe the tides were for a slightly different region, etc. New predictions always differ from previous predictions in some way, but this does not mean the predictions are useless. The resolution is that "topic" is not a precise idea. Rather, if a forecaster is successful on topics A, B, and C that are all in a natural reference class, and D is also in that reference class, then this is evidence — not incontrovertible evidence, but strong compelling evidence —that they are likely to be accurate on D too.