4 ms·
"Forecaster" here means any rando who signs up to their service and answers a question. Sometimes a large polling number does not equal a more accurate answer.
by zgluck 3y ago
"Forecaster" here means any rando who signs up to their service and answers a question.
Sometimes a large polling number does not equal a more accurate answer.
- saulmunn 3y agothis isn't quite true -- on metaculus, accounts that have a history of forecasting things well are weighted more heavily
- zgluck 3y agoSISO (Shit In, Shit Out) still applies. You guys need a high quality user base with domain knowledge, at least as a seed. There is no proof that you have that at the moment. Edit: Okay, that track record page avionical posted in a separate comment is actually a bit convincing now that I dig deeper into it. :-) I suppose that for e.g. AI/AGI a weakness could be that the estimates for most of the users have been short term (a few years at most) but the AGI estimates are 8-17 years away. Those estimates are a lot harder to do make and hence surely a lot less accurate.
- avionical 3y agoWhat would be good evidence of a high-quality user base with the relevant skills? A transparent, well-calibrated track record?
- avionical 3y agoAnyone can forecast, sure. But there's a large body of research on the accuracy of aggregated forecasts and on the ability of forecasters to become more accurate with practice. (Thinking here in particular of work by Mellers & Tetlock.) Metaculus provides a transparent track record of community forecasts here: https://www.metaculus.com/questions/track-record/ https://www.metaculus.com/questions/track-record/ It's very difficult for any one person to consistently beat the community.