3 ms·

If the results are 65% accurate, could one not conduct a meta-study to achieve the requisite level of confidence?

• gwern 12y ago
  They don't have that many cause-effect pairs in their dataset (~80, IIRC), so it could be considered insufficient data depending on how skeptical you are. But you don't really need to do a meta-analysis: the cause-effect pair collection is available, so if you dig up more, you can simply rerun the algorithms and see what the new results are with more pairs available. I don't know how hard it would be to collect more examples: on the one hand, any randomized experiment should be a source of such examples, but on the other hand, you might want datasets large enough to definitively establish a causal effect before you go using it as a ground-truth/benchmark in statistical studies like this (and larger datasets tend to be harder to get the individual datapoints for).