3 ms·
> Second, the growing use of meta-analyses, which combine results across studies, has started to make clear that the tendency not to publish negative results gi
by bunderbunder 8y ago
> Second, the growing use of meta-analyses, which combine results across studies, has started to make clear that the tendency not to publish negative results gives misleading impressions.
Mark Crislip, host of PusCast, has made a few good wisecracks on this subject. Perhaps the most poetic is (paraphrasing here): "When you mix cow pie and apple pie, you do not make the cow pie better. You make the apple pie worse."
- theobon 8y agoMeta-analysis is a sound principle but undermined by selection bias. This is why every meta-analysis includes selection criteria for which studies are incorporated into the analysis. Unfortunately publication bias is systemic and biases the results. The quote referenced is poetic and pithy but I fail to understand how it applies to meta-analysis and selection bias. What is the cow pie in this scenario?
- bunderbunder 8y agoThe cow pie is biased work. The root problem with using meta-analysis is that it just wasn't designed to work with real-world science. It generally assumes that it's being applied to an unbiased sample of unbiased results. It's now pretty well understood that the published literature is really a biased sample of (oftentimes) biased results. No amount of selection criteria can fix that; the best you can hope for is that they will yield a biased sample of unbiased results. I'm no expert on health science, I'm just taking potshots from the peanut gallery, but I'd guess it's pretty much always better to ditch the shiny mathematical bauble and its false promise of providing an easy, simple, objective answer to an inherently subtle and complex problem, roll up your sleeves, and get to work on a proper systematic review.