4 ms·

The problem with this is that it seems likely to be biased against unexpected results or results that contradict the dominant theory. The old saying, "Science a

• mrjaeger 11y ago
  I'm sure there is some natural human bias against something totally unexpected being correct, but these people also read the papers. I imagine people would be more concerned with methodological flaws and the like. Also if many unexpected results continue to be verified, the market should shift to properly value those types of studies.
• michael_fine 11y ago
  That's a feature, not a bug, I believe. Results that contradict the dominant theory are in general much less likely than one's that confirm it or extend it, and as such will have much lower odds. As a result, if you have significant evidence that such a result is true, then there is a large upside in betting on it, enough to outweigh the unlikelihood.
  • sharp11 11y ago
    That's interesting. You're looking at it from the point of view of the bettor. My concern is just that non-dominant lines of inquiry are already intensely selected against and this just institutionalizes that bias. But I see your point that the lopsided upside incentivizes contrarians. That's very interesting .. I have to think about that! [Edited as enlightenment dawns ...]
    • savanaly 11y ago
      That's as it should be, though. Extraordinary claims require extraordinary evidence, and studies should be regarded more skeptically the more surprising their results are.
• yummyfajitas 11y ago
  If you are correct you can make money by betting contrary to the bias. Science advances one funeral at a time. The stock market accelerates the process by separating fools from their money.
  • Alex3917 11y ago
    > If you are correct you can make money by betting contrary to the bias. This betting market only covers reproducibility, not truth. But since most reproducible findings are still false, betting contrary to bias is unlikely to work.
    • yummyfajitas 11y ago
      Could you give some toy example to illustrate this? I honestly don't understand what you are trying to claim. Toy example to illustrate my claim: Suppose a given finding has a probability p of replicating, but the biased market estimates q < p. This means that you must spend $q and if the experiment replicates you'll earn $1.00. On average you'll win $p from these bets - $1.00 exactly p of the time. Your net winnings will be $p-$q. As long as your theory more accurately predicts the replication probability, you make money.
      • Alex3917 11y ago
        As an example, what if the statistical analysis for a study was wrong? In that case the study may well be replicable, but the underlying claim being made by the study is still false. At least in the context of the phrase 'science advances on funeral at a time', usually folks are talking about studies that are wrong for reasons other than replicability. (Because in general it's pretty easy to convince folks that something is false if it can't be replicated, but much more difficult to convince folks that there is some deeper methodological or epistemological issue.)
        • yummyfajitas 11y ago
          Given that replicability would presumably be about predicting observable statistics, an incorrect calculation wouldn't change it. I.e., observe a bernoulli trial and see 60 successes in 100 trials. Plan a followup study which will repeat the bernoulli trial 100x - predictions would be on the # of successes in the followup, not some statistical analysis. The phrase "science advances one funeral at a time" is somewhat orthogonal to this idea - it tends to be about older scientists being unwilling to accept new theories in spite of repeated successes. Such scientists would lose their money if they bet or be ignored if they didn't.
• eli_gottlieb 11y ago
  Actual results with low likelihood tell us a lot. Results with high likelihood tell us a little.