4 ms·
The Cochrane Collaboration is a noble effort but its impact is limited in practice. There is a huge incentive to produce papers in the modern medical system, an
by rscho 6y ago
The Cochrane Collaboration is a noble effort but its impact is limited in practice. There is a huge incentive to produce papers in the modern medical system, and I'm in a good place to know that you can make the data say whatever you want with techniques such as meta analysis, which incidentally is a big part of Cochrane studies. Meta analysis as a field has grown trendemously because who doesn't want to publish high impact papers without having to gather any new data, right? Publication of end-to-end single-command build reproducible research should be the only one allowed at this point, with everything accessible to the reader, including data. I've been doing shit medical science for ten years already because of hierarchical pressure, and I'd love a change of course.
- Tenoke 6y agoAs far as I can tell the Cochrane meta-analyses are usually of above average quality, pretty honest, don't have particular reason or evidence for bias in any direction and are conducted fairly well. Do you have any specific examples of problems in their meta-analyses?
- rscho 6y agoFirstly, I have a problem with meta analysis in general and the sheer volume of meta papers tends to drown the Cochrane papers. Additionally, I challenge the judgment calls about honesty of authors, be it the source trial authors or the meta authors. There is a _really_ large interpretation gap between what is written and what really happened depending on how you encode the data into the dataset and the analysis model. I am a clinical researcher. And TBH, I think it's not really possible to conduct truthful and fundamentally believable clinical research given the current social and technological environments of academic research. I'm far from the only one slaving for professors who don't know what they're doing. If I did not have a family to feed, I would not accept doing the work I am doing right now.
- neuronic 6y agoMeta-analysis is still very valuable though because you would have a hard time finding significant disagreements between studies otherwise. Of course it comes with the benefit of not having to produce novel data but IMO that does not absolve such studies from merit and impact.
- rscho 6y ago> that does not absolve such studies from merit and impact Not completely, but is it justified that meta analysis papers are among the highest-impact things you can produce when it is known that the chance of type I error is particularly high? Really, I think we should strive to improve data retrieval capabilities in our healthcare systems to conduct trials based on hard data as extensively as possible instead of bickering over the dubious scientific interpretation of clinical facts gathered, analyzed and interpreted by people having little to no scientific education.