4 ms·

Expecting medicine to work with any doctor on any patient is what made doctors stop making excuses for bad medicine - "It works when I do it" or "It works if yo

by wisty 6y ago

Expecting medicine to work with any doctor on any patient is what made doctors stop making excuses for bad medicine - "It works when I do it" or "It works if you use it on the right patient" when they're just talking pure quackery. It's kind of similar in management, education, sociology, software development, and plenty of other not-so-well respected fields (a manager might get paid well, but they aren't respected like doctors are). Experts love to say that "every case is different" because then they can always defend their pet crackpot theory with "well, it will work if the right person does it properly in the right case" (which is kind of a truism). Yes, medical experts will take things from "medicine X works well" to "but we found certain interactions / contraindications where it won't work so well, and a few cases where it works better", but this is different - it's expected that they're specific.

eru 6y ago
That reminds me of the social sciences (and their replication crisis). There's nothing wrong with effects that only apply very narrowly, or medical techniques that only work for one person. What is wrong is to use the defence that 'every case is different' and 'well, it will work if the right person does it properly in the right case' one the one hand, but claim universal applicability of your techniques or results on the other. See also motte-and-bailey fallacy.
- phreeza 6y ago
  Is it even possible in today's evidence based framework to consider a treatment that only works for a single patient? I think you'd need at least an entire class of such treatments applying to a group of patients to do any kind of RCT or similar study.
  - eru 6y ago
    Well, if RCT is your only standard of evidence, then you need multiple people almost by definition. But it's not the only standard of evidence used. Famously, there has never been an RCT for parachutes. (Until recently that is, see https://healthjournalism.org/blog/2019/02/breakthrough-research-reveals-parachutes-dont-prevent-death-when-jumping-from-a-plane/ https://healthjournalism.org/blog/2019/02/breakthrough-resea... )
- barry-cotter 6y ago
  > That reminds me of the social sciences (and their replication crisis). It’s not a social science replication crisis. One of the central papers in this rediscovery of bad statistical practice was Ioannides “ Why Most Published Research Findings Are False” https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... and Bayer’s internal research finds most preclinical cancer studies don’t replicate. http://www.readthehook.com/103149/junk-science-most-preclinical-cancer-studies-dont-replicate http://www.readthehook.com/103149/junk-science-most-preclini...
- thaumasiotes 6y ago
  > There's nothing wrong with effects that only apply very narrowly, or medical techniques that only work for one person. There's no way to tell the difference between a technique that only works for one person and a technique that works for zero people.
jychang 6y ago
The issue is that this only works in some cases where the root cause is well understood. "Bacterial infection, give this antibiotic which kills the bacteria". In other cases, the root cause of an issue is obscured under a complex system. The current cutting-edge pharmacodynamics understanding of depression is that "it is an issue with serotonin/norepinephrine/dopamine/etc... except sometimes it isn't". This leads to situations where you have psychiatrists prescribing a few SSRIs to treat someone's depression, except it sometimes doesn't work with the patient and they're not sure why, in which case they try atypical antipsychotics, except that sometimes doesn't work and they're not sure why, so they try NDRIs, which sometimes doesn't work, so they try MAOIs, except sometimes that doesn't work, in which case they try tricyclics, unless that doesn't work... you get the idea. This "guess and check" method is currently the legitimate scientific (not quackery) method that psychiatrists are working with. Unfortunately, since fields such as brain chemistry is immensely complex, "it works if you use it on the right patient" is still a legitimate scientific response in many situations.

by wisty 6y ago

eru 6y ago

That reminds me of the social sciences (and their replication crisis). There's nothing wrong with effects that only apply very narrowly, or medical techniques that only work for one person. What is wrong is to use the defence that 'every case is different' and 'well, it will work if the right person does it properly in the right case' one the one hand, but claim universal applicability of your techniques or results on the other. See also motte-and-bailey fallacy.

phreeza 6y ago
Is it even possible in today's evidence based framework to consider a treatment that only works for a single patient? I think you'd need at least an entire class of such treatments applying to a group of patients to do any kind of RCT or similar study.
- eru 6y ago
  Well, if RCT is your only standard of evidence, then you need multiple people almost by definition. But it's not the only standard of evidence used. Famously, there has never been an RCT for parachutes. (Until recently that is, see https://healthjournalism.org/blog/2019/02/breakthrough-research-reveals-parachutes-dont-prevent-death-when-jumping-from-a-plane/ https://healthjournalism.org/blog/2019/02/breakthrough-resea... )
barry-cotter 6y ago
> That reminds me of the social sciences (and their replication crisis). It’s not a social science replication crisis. One of the central papers in this rediscovery of bad statistical practice was Ioannides “ Why Most Published Research Findings Are False” https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... and Bayer’s internal research finds most preclinical cancer studies don’t replicate. http://www.readthehook.com/103149/junk-science-most-preclinical-cancer-studies-dont-replicate http://www.readthehook.com/103149/junk-science-most-preclini...
thaumasiotes 6y ago
> There's nothing wrong with effects that only apply very narrowly, or medical techniques that only work for one person. There's no way to tell the difference between a technique that only works for one person and a technique that works for zero people.

jychang 6y ago

The issue is that this only works in some cases where the root cause is well understood. "Bacterial infection, give this antibiotic which kills the bacteria". In other cases, the root cause of an issue is obscured under a complex system. The current cutting-edge pharmacodynamics understanding of depression is that "it is an issue with serotonin/norepinephrine/dopamine/etc... except sometimes it isn't". This leads to situations where you have psychiatrists prescribing a few SSRIs to treat someone's depression, except it sometimes doesn't work with the patient and they're not sure why, in which case they try atypical antipsychotics, except that sometimes doesn't work and they're not sure why, so they try NDRIs, which sometimes doesn't work, so they try MAOIs, except sometimes that doesn't work, in which case they try tricyclics, unless that doesn't work... you get the idea. This "guess and check" method is currently the legitimate scientific (not quackery) method that psychiatrists are working with. Unfortunately, since fields such as brain chemistry is immensely complex, "it works if you use it on the right patient" is still a legitimate scientific response in many situations.