3 ms·

I have always had a huge problem with the non-reproducible nature of medical research and its acceptance within the field. Being in medicine, every day you hear

by kyro 12y ago

I have always had a huge problem with the non-reproducible nature of medical research and its acceptance within the field. Being in medicine, every day you hear a physician citing some study from 15 years ago conducted on a sample size of 20/50/100 or so patients as a way to justify their clinical decisions. And it always worries me that we tend to put so much faith in these "landmark" studies, as if their findings are somehow legitimate and true because statistical significance was reached at least once, and seem to forget all of the misaligned incentives and game-playing that goes on in research, and the amount of data burying by the FDA, all that almost certainly influence and manipulate data, publication, etc in a negative way. I tend to take the majority of medical research with a grain of salt, for the reasons listed here and in the article, unless there's some very convincing meta-analysis or successfully reproduced evidence. Call me overly cynical, but that we calculate a parameter or administer a drug or change our methods because of some article you read last month in NEJM is beyond bogus.

biomcgary 12y ago
I'm a computational biologist working at the intersection of basic research and medical research. I would recommended more than one grain of salt with medical research. Medically trained folks often fail to understand study design and are rarely rigorous thinkers. The heavy emphasis on memorization in medical school is a substantial part of the problem. And, those are problems even before perverse incentives kick in.
- pcrh 12y ago
  In medical research there is also the additional handicap of career structure. The extremely hierarchical nature of medical careers, and the substantial financial rewards for "playing along", mean that there are strong disincentives to rock the boat by criticizing the hypotheses of senior members. This often results in byzantine twists of logic being used to justify continuing with particular lines of enquiry, such as the amyloid hypothesis in Alzheimer's disease (my field).
DMac87 12y ago
Ditto for academic finance - so many sketchy results get advertised, and billions if not trillions of dollars get invested based on false or uncertain hypotheses... Sometimes I feel that 90% of my job is to be skeptical of all results, and convince managers not to do anything stupid - a sort of "first, do no harm" philosophy.

by kyro 12y ago

biomcgary 12y ago

I'm a computational biologist working at the intersection of basic research and medical research. I would recommended more than one grain of salt with medical research. Medically trained folks often fail to understand study design and are rarely rigorous thinkers. The heavy emphasis on memorization in medical school is a substantial part of the problem. And, those are problems even before perverse incentives kick in.

pcrh 12y ago
In medical research there is also the additional handicap of career structure. The extremely hierarchical nature of medical careers, and the substantial financial rewards for "playing along", mean that there are strong disincentives to rock the boat by criticizing the hypotheses of senior members. This often results in byzantine twists of logic being used to justify continuing with particular lines of enquiry, such as the amyloid hypothesis in Alzheimer's disease (my field).

DMac87 12y ago

Ditto for academic finance - so many sketchy results get advertised, and billions if not trillions of dollars get invested based on false or uncertain hypotheses... Sometimes I feel that 90% of my job is to be skeptical of all results, and convince managers not to do anything stupid - a sort of "first, do no harm" philosophy.