4 ms·
This submission was probably sparked by the recent prediction market results from the Social Science Replication Project. In this study, a prediction market wit
by tedsanders 8y ago
This submission was probably sparked by the recent prediction market results from the Social Science Replication Project. In this study, a prediction market with 206 traders and $20,600 of incentives was able to predict whether 21 psychology studies would replicate with pretty astounding accuracy (5/5 bets with <50% predicted did not actually replicate, and 13/16 with >50% did - and the 3 misses were predicted at like 51%, 52%, 53%).
Coverage by the Atlantic here: https://www.theatlantic.com/science/archive/2018/08/scientists-can-collectively-sense-which-psychology-studies-are-weak/568630/ https://www.theatlantic.com/science/archive/2018/08/scientis...
Coverage by 538 on previous work by the same authors: https://fivethirtyeight.com/features/how-to-tell-good-studies-from-bad-bet-on-them/ https://fivethirtyeight.com/features/how-to-tell-good-studie...
Commentary on LessWrong here: https://www.lesswrong.com/posts/7w2tCRocphK9yW8rG/unknown-knowns https://www.lesswrong.com/posts/7w2tCRocphK9yW8rG/unknown-kn...
And my own thoughts here: https://www.facebook.com/tedsanders/posts/10106092033737384 https://www.facebook.com/tedsanders/posts/10106092033737384
For reference, I was one of the top prize winners on SciCast.org, an academic science prediction market that Robin Hanson (author of linked article) was involved in.
I personally am excited by scientific prediction markets because they are a systematic way of making hidden knowledge public.