5 ms·

Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical r

by rscho 5y ago

Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else. And there's a reason to that: MDs are not scientists. We're highly-skilled workers, and more of the blue-collar kind.

dr_dshiv 5y ago
Disagree. Think your standards are too high.
- throwoutway 5y ago
  Your standards are too low. Human lives are at stake in medicine. Do you want a doctor treating you when they are misinformed by fake science?
  - dr_dshiv 5y ago
    I want doctors to not treat studies like “science sez” and instead recognize that “Science is messy”. They should use their training and experience to heal, not the latest publication.
  - webmaven 5y ago
    I'm not sure that's any worse than a doctor misunderstanding real science. Note that the root cause (unfamiliarity with the tools of science, especially probability and statistics) is largely the same.
    - rscho 5y ago
      > I'm not sure that's any worse than a doctor misunderstanding real science It is worse. A highly experienced practitioner who would've done the right thing might get led astray by new bogus research. Precisely because most practitioners don't have the ability to judge scientific work on their own.
    - throwoutway 5y ago
      There's a difference between mistakes and being maliciously tricked into making those mistakes
      - webmaven 5y ago
        Sure, there is a difference of intent, and a difference in motivating incentives, but the line is fuzzier than we like to think (eg. industries providing no-strings-attached funding to study everything but the area that has issues they wish to crowd out).
mananaysiempre 5y ago
> We're highly-skilled workers, and more of the blue-collar kind. While it’s a bit of a problem that sometime during the last century a skilled profession got equated with a research one (I remember hearing somewhere that a commercial pilot’s education costs about as much as a tenured professor’s, and it seems plausible to me), there’s also the fact that the people you call “workers” are more distanced from research than they used to be; my impression is that this is even stronger among engineers than it is among medical professionals, but I can hardly claim any deep knowledge here. MIT is originally an engineering college, so is the École polytechnique, and the École normale is a school for bureaucrats. Cauchy’s seminal work on wave propagation was presented at the French academy, had an epigraph from Virgil (“how many waves / come rolling shoreward from the Ionian sea”), and was signed by “Mr. Augustin-Louis Cauchy, road and bridge engineer” (this being the title the Polytechnique awarded him). It used to be normal (AFAIK) for articles in medical journals to be essentially “hey, look what a weird thing happened with this recent patient of mine”, and some of seminal (if not the most high-brow) mathematics journals started as “hey, solve this problem I have recently done / want to know the answer to ... if you’re not chicken to try”. What happened to all of this was, in a word, specialization—necessitated in part by the exhaustion of low-hanging fruit and raising of epistemic standards, so not entirely pointless, but I have to admit the comparative conservatism of medicine in this regard always looked more hopeful to me than the, at times, raging anti-intellectualism of engineers or the fiddly, illogical, invented world of law. Certainly medicine is more engineering than science in that it is driven more by immediate goals than by the pursuit of truth alone (not the only possible definition of science, but a useful distinction when it comes to the feeling of the field, cf the 50s adage about physics and military physics being like music and military music). And I certainly won’t say you’re wrong. But I’m sad, and I can’t help feeling the drive for industrialization has done a disservice to us, scientists and craftsmen both.
- pmoriarty 5y ago
  "raging anti-intellectualism of engineers" What makes you think engineers are anti-intellectual? Engineering is an intellectual discipline, after all.
  - 5y ago
timr 5y ago
> Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else. I couldn't agree more. Throughout the pandemic there has been an infuriating pattern of terrible research coming from medical doctors, followed by dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors". The average medical doctor has next to zero ability to evaluate a study, and is easily fooled by elaborate statistical techniques into concluding that up is down and left is right. Meanwhile, scientists and statisticians with enough experience to see the methological flaws are dismissed, and policy makers turn to MDs -- who are often wildly out of their depth -- for pronouncements on "science".
- rscho 5y ago
  > The average medical doctor has next to zero ability to evaluate a study Yes, but... > dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors" ... unfortunately, no. While it's true that most MD clinical researchers are depressingly bad, it is also true that pure scientists who never set foot in the clinics are very limited when evaluating practical aspects. So the situation cannot be trivially solved by allowing scientists to do more. In my experience, the problem has no real solution right now and I expect we'll have to wait on technological progress to allow improvements.
  - trentgreene 5y ago
    Yeah, this is the really hard part about medical research. At the individual level for those who are really qualified to do it, the incentives lean in favor of practice, where your comp is 4x-10x higher
    - rscho 5y ago
      Exactly, yes, a hundred times yes! This is the major flaw in our system. But I don't see a way of rebalancing to saner priorities. And apparently, neither do our supreme leaders.
caddemon 5y ago
I 100% agree about MDs, but believe me I've seen plenty of garbage basic bio research done by people who took a pure PhD track and ostensibly passed a statistics requirement. So I don't think it's the only issue, even though it probably is a big one.
throwoutway 5y ago
> ignorance is hurting science more than anything else It hurts more than science, it injures, kills, maims, and blinds humans. These are people's children, mothers, and fathers.
mrjangles 5y ago
What I think is hurting science more than anything else is people conflating academia with science. Academia, the process where ideas are evaluated on how popular they are, has been around for thousands of years, usually controlled by religions. Science was literally invented as a reaction against academia, and the world was better for it. I say this because while I agree with the notion of what you are saying, but you are making the same error. Real, and amazing science is being done right now at Monsanto, at Intel, and at every medical company around the world. None of these problems you mention exist in this real world of science.

by rscho 5y ago

dr_dshiv 5y ago

Disagree. Think your standards are too high.

throwoutway 5y ago
Your standards are too low. Human lives are at stake in medicine. Do you want a doctor treating you when they are misinformed by fake science?
- dr_dshiv 5y ago
  I want doctors to not treat studies like “science sez” and instead recognize that “Science is messy”. They should use their training and experience to heal, not the latest publication.
- webmaven 5y ago
  I'm not sure that's any worse than a doctor misunderstanding real science. Note that the root cause (unfamiliarity with the tools of science, especially probability and statistics) is largely the same.
  - rscho 5y ago
    > I'm not sure that's any worse than a doctor misunderstanding real science It is worse. A highly experienced practitioner who would've done the right thing might get led astray by new bogus research. Precisely because most practitioners don't have the ability to judge scientific work on their own.
  - throwoutway 5y ago
    There's a difference between mistakes and being maliciously tricked into making those mistakes
    - webmaven 5y ago
      Sure, there is a difference of intent, and a difference in motivating incentives, but the line is fuzzier than we like to think (eg. industries providing no-strings-attached funding to study everything but the area that has issues they wish to crowd out).

mananaysiempre 5y ago

> We're highly-skilled workers, and more of the blue-collar kind. While it’s a bit of a problem that sometime during the last century a skilled profession got equated with a research one (I remember hearing somewhere that a commercial pilot’s education costs about as much as a tenured professor’s, and it seems plausible to me), there’s also the fact that the people you call “workers” are more distanced from research than they used to be; my impression is that this is even stronger among engineers than it is among medical professionals, but I can hardly claim any deep knowledge here. MIT is originally an engineering college, so is the École polytechnique, and the École normale is a school for bureaucrats. Cauchy’s seminal work on wave propagation was presented at the French academy, had an epigraph from Virgil (“how many waves / come rolling shoreward from the Ionian sea”), and was signed by “Mr. Augustin-Louis Cauchy, road and bridge engineer” (this being the title the Polytechnique awarded him). It used to be normal (AFAIK) for articles in medical journals to be essentially “hey, look what a weird thing happened with this recent patient of mine”, and some of seminal (if not the most high-brow) mathematics journals started as “hey, solve this problem I have recently done / want to know the answer to ... if you’re not chicken to try”. What happened to all of this was, in a word, specialization—necessitated in part by the exhaustion of low-hanging fruit and raising of epistemic standards, so not entirely pointless, but I have to admit the comparative conservatism of medicine in this regard always looked more hopeful to me than the, at times, raging anti-intellectualism of engineers or the fiddly, illogical, invented world of law. Certainly medicine is more engineering than science in that it is driven more by immediate goals than by the pursuit of truth alone (not the only possible definition of science, but a useful distinction when it comes to the feeling of the field, cf the 50s adage about physics and military physics being like music and military music). And I certainly won’t say you’re wrong. But I’m sad, and I can’t help feeling the drive for industrialization has done a disservice to us, scientists and craftsmen both.

pmoriarty 5y ago
"raging anti-intellectualism of engineers" What makes you think engineers are anti-intellectual? Engineering is an intellectual discipline, after all.
- 5y ago

timr 5y ago

> Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else. I couldn't agree more. Throughout the pandemic there has been an infuriating pattern of terrible research coming from medical doctors, followed by dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors". The average medical doctor has next to zero ability to evaluate a study, and is easily fooled by elaborate statistical techniques into concluding that up is down and left is right. Meanwhile, scientists and statisticians with enough experience to see the methological flaws are dismissed, and policy makers turn to MDs -- who are often wildly out of their depth -- for pronouncements on "science".

rscho 5y ago
> The average medical doctor has next to zero ability to evaluate a study Yes, but... > dismissal of any/all critiques by people much more qualified to make these critiques, because "they're not doctors" ... unfortunately, no. While it's true that most MD clinical researchers are depressingly bad, it is also true that pure scientists who never set foot in the clinics are very limited when evaluating practical aspects. So the situation cannot be trivially solved by allowing scientists to do more. In my experience, the problem has no real solution right now and I expect we'll have to wait on technological progress to allow improvements.
- trentgreene 5y ago
  Yeah, this is the really hard part about medical research. At the individual level for those who are really qualified to do it, the incentives lean in favor of practice, where your comp is 4x-10x higher
  - rscho 5y ago
    Exactly, yes, a hundred times yes! This is the major flaw in our system. But I don't see a way of rebalancing to saner priorities. And apparently, neither do our supreme leaders.

caddemon 5y ago

I 100% agree about MDs, but believe me I've seen plenty of garbage basic bio research done by people who took a pure PhD track and ostensibly passed a statistics requirement. So I don't think it's the only issue, even though it probably is a big one.

throwoutway 5y ago

> ignorance is hurting science more than anything else It hurts more than science, it injures, kills, maims, and blinds humans. These are people's children, mothers, and fathers.

mrjangles 5y ago

What I think is hurting science more than anything else is people conflating academia with science. Academia, the process where ideas are evaluated on how popular they are, has been around for thousands of years, usually controlled by religions. Science was literally invented as a reaction against academia, and the world was better for it. I say this because while I agree with the notion of what you are saying, but you are making the same error. Real, and amazing science is being done right now at Monsanto, at Intel, and at every medical company around the world. None of these problems you mention exist in this real world of science.