4 ms·

The article also relies heavily on a metaanalysis, not just individual studies. Meta analyses are designed to take a large multitude of studies in the same area

The article also relies heavily on a metaanalysis, not just individual studies. Meta analyses are designed to take a large multitude of studies in the same area and ascertain an effect size. By examining everything in aggregate, this should counteract any funding bias -- or reveal it if the results get skewed (i.e. lots of positive results and lots of negative results but no middling results). Additionally, meta analyses usually consist of a "file drawer" search where they get their hands on a bunch of the unpublished papers and analyze those as well to ensure they counteract any publishing bias for printing papers that only show positive results.

WalterSear 11y ago
I don't know where you got the idea that metanalysis is mostly unpublished work. I've never seen a published meta analysis like that. Most journals wouldn't print it.
- capnrefsmmat 11y ago
  It's happened in some notable cases; for example, a major review of reboxetine, a Pfizer antidepressant, obtained unpublished data from German health regulators and demonstrated that there was no benefit over placebo. The published data gave the opposite impression. Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., et al. (2010). Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 341. doi:10.1136/bmj.c4737 But yes, it's not standard practice. Meta-analyses try to check for publication bias but they do not systematically hunt down unpublished results.
- floppydisk 11y ago
  I work first hand in a research community and I've never seen a meta analysis published that was based solely on unpublished works. However, most of the good ones consist of a mix of published and unpublished works to ascertain whether there's a real effect or whether there's been a publishing bias that doesn't reflect the total body of research. A good meta analysis will take the time to discuss their methods for selecting published works and any efforts they made to solicit unpublished works such as contacting prominent authors in the field, getting on mailing lists, etc. The good ones also tend to discuss how they analyzed the papers for selection, discuss the paper stats (i.e. we started with 200 published papers and 20 unpublished ones, after winnowing this got reduced to 130 published and 10 unpublished), and how they collapsed results to calculate effect sizes etc. Meta analyses should include as much of the published literature as possible for the research question they're attempting to answer AND they should seek out unpublished works. The meta analysis will consist primarily of published works, the unpublished works are collected to counter the presence of positive result publishing bias. This provides a fuller picture of the research field than simply relying on published works. After they've collected the articles, the researchers analyze the collected works to ascertain which ones actually address the research question(s) they're attempting to answer, they toss everything else.

WalterSear 11y ago

I don't know where you got the idea that metanalysis is mostly unpublished work. I've never seen a published meta analysis like that. Most journals wouldn't print it.

capnrefsmmat 11y ago
It's happened in some notable cases; for example, a major review of reboxetine, a Pfizer antidepressant, obtained unpublished data from German health regulators and demonstrated that there was no benefit over placebo. The published data gave the opposite impression. Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., et al. (2010). Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 341. doi:10.1136/bmj.c4737 But yes, it's not standard practice. Meta-analyses try to check for publication bias but they do not systematically hunt down unpublished results.
floppydisk 11y ago
I work first hand in a research community and I've never seen a meta analysis published that was based solely on unpublished works. However, most of the good ones consist of a mix of published and unpublished works to ascertain whether there's a real effect or whether there's been a publishing bias that doesn't reflect the total body of research. A good meta analysis will take the time to discuss their methods for selecting published works and any efforts they made to solicit unpublished works such as contacting prominent authors in the field, getting on mailing lists, etc. The good ones also tend to discuss how they analyzed the papers for selection, discuss the paper stats (i.e. we started with 200 published papers and 20 unpublished ones, after winnowing this got reduced to 130 published and 10 unpublished), and how they collapsed results to calculate effect sizes etc. Meta analyses should include as much of the published literature as possible for the research question they're attempting to answer AND they should seek out unpublished works. The meta analysis will consist primarily of published works, the unpublished works are collected to counter the presence of positive result publishing bias. This provides a fuller picture of the research field than simply relying on published works. After they've collected the articles, the researchers analyze the collected works to ascertain which ones actually address the research question(s) they're attempting to answer, they toss everything else.