7 ms·

One in ten medical treatments are supported by high-quality evidence – study

126 points by elorant 6y ago

spzb 6y ago
Spam blog republishing this article : https://theconversation.com/only-one-in-ten-medical-treatments-are-backed-by-high-quality-evidence-145224 https://theconversation.com/only-one-in-ten-medical-treatmen...
- evolve2k 6y ago
  > Please submit the original source. If a post reports on something found on another site, submit the latter. Yes please change the link to the original source as per the HN Guidelines.
MiroF 6y ago
One in ten medical treatments that have had updated reviews recently are currently lacking high-quality evidence. I think it would be worrying if every medical review had high-quality evidence. Just because it's in the Cochrane database doesn't (to my understanding) mean that it is a frequently used treatment.
- jjcon 6y ago
  Bingo - this is not one in ten medical treatments administered, it is one in ten possible medical treatments.
- projektfu 6y ago
  This is the most important comment. The title is misleading.
jvanderbot 6y ago
Medicine is old. Many uses of penicillin are not well studied, because it was the first real antibiotic and we just know it works for what it does. Nobody can get funding to study penicillin uses unless it somehow magically can cure something new.
- rsynnott 6y ago
  Also, there’d be significant ethical issues with trying to prove that some long-used treatment works. There’s really no way to get a control group.
- dehrmann 6y ago
  There was article in The Atlantic a few months ago about fluoride in water not probably works, but isn't particularly well-studied, and long-term side effects aren't really understood. At least with fluoride, it's so widespread that you'd expect any serious issues to have surfaced by now. https://www.theatlantic.com/magazine/archive/2020/04/why-fluoride-water/606784/ https://www.theatlantic.com/magazine/archive/2020/04/why-flu...
  - akimball 6y ago
    You mean like increase in autism, depression, and decrease in population mean testosterone?
- doesnotexist 6y ago
  There are 55 cochrane reviews if you search for pencillin on their site https://www.cochranelibrary.com/advanced-search https://www.cochranelibrary.com/advanced-search. Infectious Disease is one of the most academic oriented and evidence rigorous disciplines in all of medicine. As the requirements for levels of evidence are ratcheted up many treatments that are old and established are reviewed and undergo additional study to establish them under the new expectations.
newsbinator 6y ago
> 22 percent had very low-quality evidence.
ncmncm 6y ago
If this list includes depression treatments, they are examples of treatments for which "high-quality evidence" of the sort that is demanded is impossible. The only way to know whether a person has the version of the disorder that responds to a particular drug is to administer that drug and see. Psychiatrists try prescribing different drugs until they hit one that works, or give up. Imagine designing a randomized-controlled-trial for that. If you need help: first, select a group already using the drug. Split them up and give half a placebo. See which get worse. But first, find someone unethical enough to run it, and admits it.
- jeffreyrogers 6y ago
  That makes a big assumption that there are multiple types of depression caused by different chemical imbalances. Not saying that's a wrong assumption (e.g. for cancer the analogous assumption that many different mutations can lead to similar cancers is true), but it is a big assumption, and there are plenty of psychologists who would disagree with it.
  - doesnotexist 6y ago
    "Scientifically speaking, there never was a network of validated hypotheses capable of sustaining a full-blown, global chemical imbalance theory of mental illness. Moreover-and here we come back to Myth 2-psychiatry as a profession and medical specialty never endorsed such a bogus “theory,” when judged by its professional organizations, its peer-reviewed publications, its standard textbooks, or its official pronouncements." https://www.psychiatrictimes.com/view/debunking-two-chemical-imbalance-myths-again https://www.psychiatrictimes.com/view/debunking-two-chemical... In the general public, there's a logical fallacy that people seem to fall for when thinking about psychopharmacology. It seems that when it is learned that serotonin re-uptake inhibitors (SSRIs)are used to treat patients with depression it is followed with some mechanistic explanation of a "chemical imbalance" resulting from "not enough serotonin." Yet people don't make this mistake when talking about opiods or pain killers. If you break your arm and the doctor prescribes Codeine we don't come to the conclusion that "you broke your arm and all your endogenous opoids came spilling out so we had to replace it with Codeine." Although Tylenol/paracetamol are useful for treating pain, we don't hear a lot of talk about Tylenol deficiencies.
otabdeveloper4 6y ago
Well, yes. Experimenting on humans is unethical and (mostly) illegal.
- pessimizer 6y ago
  Treatments with bad evidence are no better than experiments. Treatments also only become acceptable over time, after we observe that we're probably not killing or seriously hurting people. Of course, sometimes we just rationalize it when the treatments actually are killing people, if the people profiting from them aren't actually engaging in outright concealment and fraud.
AshamedCaptain 6y ago
But, whenever there is a single study claiming something new, it quickly goes into mass media (including HN), and calls for "skepticism until this is reproduced a couple more times and the positive results significantly outnumber the negative ones" are quickly downvoted.
ashton314 6y ago
Neil Postman has a chapter about how medical technology ha changed what people expect from medical treatment in his book _Technopoly_. The short of it is that we’ve become reliant on technological solutions that we demand the maximum treatment available land we litigate when the doctor doesn’t order the maximum number of tests or prescribe the maximum treatment, rather than rely on the doctor carefully listening to the patient and meting out more protracted treatments. I think that it’s good that researchers try every possible avenue for research, but I think our obsession with increasingly technological “solutions” is driving an unhealthy environment of mistakes in treatments in the interest of delivering the maximum possible.
csr86 6y ago
I have had failed ankle surgery and have read some research papers on the procedure that was done to me. Often, the method was asessed by asking patients to score their situation before and after the surgery (e.g one year later). For sure, many people try to be positive and give too optimistic scores. At least I felt it hard to admit that the costly procedure had failed and saying it to my surgeon didn't feel easy. What I fear is, there are many research papers done using patient questionaire and giving us biased results
- neolog 6y ago
  What would be better than patient questionnaire?
  - Scoundreller 6y ago
    Functional assessment? Range of motion, strength, etc.
    - Brakenshire 6y ago
      You need devices which can objectively measure the strength of something like an ankle in multiple vectors of motion. And then algorithms which can combine the data into a meaningful index.
      - chrisco255 6y ago
        That doesn't sound very difficult. You could measure the angle that someone could extend before feeling pain, or ensure the ankle could apply a certain amount of pressure before the patient feels pain, etc.
        nradov 6y ago
        It would be very difficult to bring that to market as an FDA certified medical device. And then repeat the process for every other joint.
akimball 6y ago
Very few of them are, because each case is different. Another way to put this: EBM is only good for about 1 in 10 treatments. Great for writing papers, though.
TheRealPomax 6y ago
Cool, what was that number 5 years ago? 10? 20? 50? Are we making progress or have we only just now established a baseline and the fact that we now know it's "1 in 10" is a number that lets us draw exactly zero conclusions with respect to whether that's good or bad, and more importantly, better or worse than before?
strstr 6y ago
This number is shocking, but it is not weighted by usage. Just because a medical treatment is a treatment doesn’t mean it’s in common use. On top of that, I find it 100% believable that doctors use understudied treatments. The reverse makes a lot more sense. Treatments get studied after people have tried them, and believe they warrant further study and more frequent usage. Consider coronavirus treatments, at first we had very little idea what to do, beyond inferring based on past virus with a similar symptom profile and reasoning from first principles. Later on, we slowly but surely identified effective treatments (corticosteroids/prone position/...) after people tried them. Even later, we had descriptive studies of past effectiveness. I assume most treatments go through a similar progression (covid treatments go through this progression on fast forward). Medical studies are slow and retrospective. Fundamentally that gets you this 1/10 figure.
askvictor 6y ago
This is a bit like the dental floss thing of a few years ago - someone suddenly realised there was no evidence backing the use of dental floss; cue media frenzy that dental floss is useless. No it isn't; just no-one bothered to study it. I'm sure that there isn't any high-quality evidence that 'removing your hand from a hot stove decreases burn injuries.' More evidence is a good thing, and is needed in many cases, but and there is also a heap of questionable medical treatment out there, perpetuated by the status quo.
- Obi_Juan_Kenobi 6y ago
  That is not an accurate summation of the evidence. 11 publications were used to analyze the efficacy of dental floss, a some dozens more for other inter-dental cleaning methods. https://onlinelibrary.wiley.com/doi/full/10.1111/jcpe.12363 https://onlinelibrary.wiley.com/doi/full/10.1111/jcpe.12363 Characterizing this as 'absence of evidence' is incorrect; the specific issue has been investigated by several independent investigators and consistently fails to find evidence of efficacy. Even with much higher sampling and statistical power, if any effect was observed, it would likely be well below anything that could justify flossing as a health recommendation, especially over IDBs.
  - askvictor 6y ago
    I stand corrected.
phkahler 6y ago
They talk about blind trials being better, but then: >> An exercise trial cannot be "blinded": anyone doing exercise will know they are in the exercise group... How exactly can knowing which group you're in cause a placebo effect or any other relevant effect in that case?
- jdmcnugent 6y ago
  Blinding is a way to actually reveal placebo effects, and double blinding (blinding the examiner to which group the subject is in) helps negate observer bias.

126 points by elorant 6y ago

spzb 6y ago

Spam blog republishing this article : https://theconversation.com/only-one-in-ten-medical-treatments-are-backed-by-high-quality-evidence-145224 https://theconversation.com/only-one-in-ten-medical-treatmen...

evolve2k 6y ago
> Please submit the original source. If a post reports on something found on another site, submit the latter. Yes please change the link to the original source as per the HN Guidelines.

MiroF 6y ago

One in ten medical treatments that have had updated reviews recently are currently lacking high-quality evidence. I think it would be worrying if every medical review had high-quality evidence. Just because it's in the Cochrane database doesn't (to my understanding) mean that it is a frequently used treatment.

jjcon 6y ago
Bingo - this is not one in ten medical treatments administered, it is one in ten possible medical treatments.
projektfu 6y ago
This is the most important comment. The title is misleading.

jvanderbot 6y ago

Medicine is old. Many uses of penicillin are not well studied, because it was the first real antibiotic and we just know it works for what it does. Nobody can get funding to study penicillin uses unless it somehow magically can cure something new.

rsynnott 6y ago
Also, there’d be significant ethical issues with trying to prove that some long-used treatment works. There’s really no way to get a control group.
dehrmann 6y ago
There was article in The Atlantic a few months ago about fluoride in water not probably works, but isn't particularly well-studied, and long-term side effects aren't really understood. At least with fluoride, it's so widespread that you'd expect any serious issues to have surfaced by now. https://www.theatlantic.com/magazine/archive/2020/04/why-fluoride-water/606784/ https://www.theatlantic.com/magazine/archive/2020/04/why-flu...
- akimball 6y ago
  You mean like increase in autism, depression, and decrease in population mean testosterone?
doesnotexist 6y ago
There are 55 cochrane reviews if you search for pencillin on their site https://www.cochranelibrary.com/advanced-search https://www.cochranelibrary.com/advanced-search. Infectious Disease is one of the most academic oriented and evidence rigorous disciplines in all of medicine. As the requirements for levels of evidence are ratcheted up many treatments that are old and established are reviewed and undergo additional study to establish them under the new expectations.

newsbinator 6y ago

> 22 percent had very low-quality evidence.

ncmncm 6y ago

If this list includes depression treatments, they are examples of treatments for which "high-quality evidence" of the sort that is demanded is impossible. The only way to know whether a person has the version of the disorder that responds to a particular drug is to administer that drug and see. Psychiatrists try prescribing different drugs until they hit one that works, or give up. Imagine designing a randomized-controlled-trial for that. If you need help: first, select a group already using the drug. Split them up and give half a placebo. See which get worse. But first, find someone unethical enough to run it, and admits it.

jeffreyrogers 6y ago
That makes a big assumption that there are multiple types of depression caused by different chemical imbalances. Not saying that's a wrong assumption (e.g. for cancer the analogous assumption that many different mutations can lead to similar cancers is true), but it is a big assumption, and there are plenty of psychologists who would disagree with it.
- doesnotexist 6y ago
  "Scientifically speaking, there never was a network of validated hypotheses capable of sustaining a full-blown, global chemical imbalance theory of mental illness. Moreover-and here we come back to Myth 2-psychiatry as a profession and medical specialty never endorsed such a bogus “theory,” when judged by its professional organizations, its peer-reviewed publications, its standard textbooks, or its official pronouncements." https://www.psychiatrictimes.com/view/debunking-two-chemical-imbalance-myths-again https://www.psychiatrictimes.com/view/debunking-two-chemical... In the general public, there's a logical fallacy that people seem to fall for when thinking about psychopharmacology. It seems that when it is learned that serotonin re-uptake inhibitors (SSRIs)are used to treat patients with depression it is followed with some mechanistic explanation of a "chemical imbalance" resulting from "not enough serotonin." Yet people don't make this mistake when talking about opiods or pain killers. If you break your arm and the doctor prescribes Codeine we don't come to the conclusion that "you broke your arm and all your endogenous opoids came spilling out so we had to replace it with Codeine." Although Tylenol/paracetamol are useful for treating pain, we don't hear a lot of talk about Tylenol deficiencies.

otabdeveloper4 6y ago

Well, yes. Experimenting on humans is unethical and (mostly) illegal.

pessimizer 6y ago
Treatments with bad evidence are no better than experiments. Treatments also only become acceptable over time, after we observe that we're probably not killing or seriously hurting people. Of course, sometimes we just rationalize it when the treatments actually are killing people, if the people profiting from them aren't actually engaging in outright concealment and fraud.

AshamedCaptain 6y ago

But, whenever there is a single study claiming something new, it quickly goes into mass media (including HN), and calls for "skepticism until this is reproduced a couple more times and the positive results significantly outnumber the negative ones" are quickly downvoted.

ashton314 6y ago

Neil Postman has a chapter about how medical technology ha changed what people expect from medical treatment in his book _Technopoly_. The short of it is that we’ve become reliant on technological solutions that we demand the maximum treatment available land we litigate when the doctor doesn’t order the maximum number of tests or prescribe the maximum treatment, rather than rely on the doctor carefully listening to the patient and meting out more protracted treatments. I think that it’s good that researchers try every possible avenue for research, but I think our obsession with increasingly technological “solutions” is driving an unhealthy environment of mistakes in treatments in the interest of delivering the maximum possible.

csr86 6y ago

I have had failed ankle surgery and have read some research papers on the procedure that was done to me. Often, the method was asessed by asking patients to score their situation before and after the surgery (e.g one year later). For sure, many people try to be positive and give too optimistic scores. At least I felt it hard to admit that the costly procedure had failed and saying it to my surgeon didn't feel easy. What I fear is, there are many research papers done using patient questionaire and giving us biased results

neolog 6y ago
What would be better than patient questionnaire?
- Scoundreller 6y ago
  Functional assessment? Range of motion, strength, etc.
  - Brakenshire 6y ago
    You need devices which can objectively measure the strength of something like an ankle in multiple vectors of motion. And then algorithms which can combine the data into a meaningful index.
    - chrisco255 6y ago
      That doesn't sound very difficult. You could measure the angle that someone could extend before feeling pain, or ensure the ankle could apply a certain amount of pressure before the patient feels pain, etc.
      - nradov 6y ago
        It would be very difficult to bring that to market as an FDA certified medical device. And then repeat the process for every other joint.

akimball 6y ago

Very few of them are, because each case is different. Another way to put this: EBM is only good for about 1 in 10 treatments. Great for writing papers, though.

TheRealPomax 6y ago

Cool, what was that number 5 years ago? 10? 20? 50? Are we making progress or have we only just now established a baseline and the fact that we now know it's "1 in 10" is a number that lets us draw exactly zero conclusions with respect to whether that's good or bad, and more importantly, better or worse than before?

strstr 6y ago

This number is shocking, but it is not weighted by usage. Just because a medical treatment is a treatment doesn’t mean it’s in common use. On top of that, I find it 100% believable that doctors use understudied treatments. The reverse makes a lot more sense. Treatments get studied after people have tried them, and believe they warrant further study and more frequent usage. Consider coronavirus treatments, at first we had very little idea what to do, beyond inferring based on past virus with a similar symptom profile and reasoning from first principles. Later on, we slowly but surely identified effective treatments (corticosteroids/prone position/...) after people tried them. Even later, we had descriptive studies of past effectiveness. I assume most treatments go through a similar progression (covid treatments go through this progression on fast forward). Medical studies are slow and retrospective. Fundamentally that gets you this 1/10 figure.

askvictor 6y ago

This is a bit like the dental floss thing of a few years ago - someone suddenly realised there was no evidence backing the use of dental floss; cue media frenzy that dental floss is useless. No it isn't; just no-one bothered to study it. I'm sure that there isn't any high-quality evidence that 'removing your hand from a hot stove decreases burn injuries.' More evidence is a good thing, and is needed in many cases, but and there is also a heap of questionable medical treatment out there, perpetuated by the status quo.

Obi_Juan_Kenobi 6y ago
That is not an accurate summation of the evidence. 11 publications were used to analyze the efficacy of dental floss, a some dozens more for other inter-dental cleaning methods. https://onlinelibrary.wiley.com/doi/full/10.1111/jcpe.12363 https://onlinelibrary.wiley.com/doi/full/10.1111/jcpe.12363 Characterizing this as 'absence of evidence' is incorrect; the specific issue has been investigated by several independent investigators and consistently fails to find evidence of efficacy. Even with much higher sampling and statistical power, if any effect was observed, it would likely be well below anything that could justify flossing as a health recommendation, especially over IDBs.
- askvictor 6y ago
  I stand corrected.

phkahler 6y ago

They talk about blind trials being better, but then: >> An exercise trial cannot be "blinded": anyone doing exercise will know they are in the exercise group... How exactly can knowing which group you're in cause a placebo effect or any other relevant effect in that case?

jdmcnugent 6y ago
Blinding is a way to actually reveal placebo effects, and double blinding (blinding the examiner to which group the subject is in) helps negate observer bias.