5 ms·

In medicine, observational evidence is actually better and far more ethical than the RCT. (Which simply dooms the terminally ill to fake treatment.) You just

In medicine, observational evidence is actually better and far more ethical than the RCT. (Which simply dooms the terminally ill to fake treatment.) You just need large datasets and an agile culture that's responsive to new input. Don't forget that RCTs are very far from perfect and issues -- sometimes literally fatal issues -- have later turned up via observational evidence in large cohorts. Vioxx, for instance. Many others. I believe, without the tiniest shred of doubt, that the only trials drugs need to go through are initial safety/toxicity trials (phases 0/1) and that everything else would be much better left to access+observation.

taeric 4mo ago
This feels off. In medicine, any evidence can also be blinded by confounding factors that are far easier to miss without adding specific controls. Really, in any field this will be the case. Should we demand an RCT before we accept evidence? Of course not. At some point you do have to make a choice on things. And it should be noted that most drugs do have early cutoff criteria if the evidence is strong enough that it is working. It isn't like people are wanting to withhold good treatments from the world. Adding controls and randomizing them, though, has proven to be highly effective at helping progress.
- A_D_E_P_T 4mo ago
  > "This feels off. In medicine, any evidence can also be blinded by confounding factors that are far easier to miss without adding specific controls. Really, in any field this will be the case." If you have enough data, you can smooth out individual fluctuations due to things like drug interactions, non-compliance, etc. (And indeed you might discover drug interactions!) Observational trials ultimately mirror how drugs are used in the real world. > "Adding controls and randomizing them, though, has proven to be highly effective at helping progress." I would argue just the opposite. Demands for increasingly byzantine trials have ballooned the costs associated with drug development, and have slowed things to a crawl. There's a reason the field's golden age was in the 1940s and 1950s, and it's not just "low hanging fruit." Today nobody in their right mind wants to work in drug development when they could work in tech or even finance.
  - taeric 4mo ago
    Right, but you are just relying on a different form of random, there. The whole point of making controls and then building experiments on changing them, is to get more power from fewer observations. No? Again, it is off to think that one is automatically superior to the other. Certainly to the exclusion of the other. And that is what feels off with the framing of the parent post. I am perfectly fine saying you should use both observational and controlled trials. But I think it is also wrong to think you don't have to build experiments to test interventions. This is why you put metrics in your service code. So that you can observe them behave and look for things to change. This is also why you do test cases on your code, so that you can specifically target your change. Now, I fully back the idea that just A/B testing something doesn't automatically mean you learn something true. But neither does observing a strong outcome on uncontrolled data.
    - estearum 4mo ago
      Yeah, GP is basically saying: "Large controlled experiments are costly and can hurt people who opt-in to informed consent. Instead, we should do significantly, significantly larger experiments, with undefined success/failure conditions, and no informed consent." Insane opinion
  - RandomLensman 4mo ago
    How do you get enough data? If, for example, you need a lot of people in the sample, that might not be so easy. In the abstract, should it not come done to what is the best experimental design for each case?
  - deleted 4mo ago
    [deleted]
  - __alexs 4mo ago
    You are simultaneously arguing for a more complex and nuanced testing approach that demands much higher quantities of data as a result, and also against RCTs, which perhaps rightly you've identified as having suffered from the same kind of cost disease as all other health care in the USA. I can't help but feel like you've identified the wrong root cause here.
naasking 4mo ago
> Which simply dooms the terminally ill to fake treatment. I wish people would stop saying this. First, controls aren't necessarily "fake treatment", they are often compared to other standard treatments. Second, the treatment being tested can actually harm the patient more, therefore the people receiving your alleged "fake treatment" can actually come out better off. Which is the "fake treatment" now? I don't disagree with your final point, but mainly with this increasingly pervasive and wrong framing of RCTs.
- FeteCommuniste 4mo ago
  Yes. Potentially life-saving drugs in RCTs are usually compared to "standard of care," i.e. already-approved treatments, not to placebos.
  - qsera 4mo ago
    So if both the standard of care as well as the drug being trials have the same side effects, would they be included in the package insert/drug literature? Because the incidence rate of it would be same for both the groups?
timr 4mo ago
> In medicine, observational evidence is actually better and far more ethical than the RCT. (Which simply dooms the terminally ill to fake treatment.) This is just nonsense. First, everyone in a trial is informed of the situation. It's not "unethical" unless you lie about it. If you participate in a trial, you do so knowing that you might not get the experimental drug. It's a selfless, honorable thing to do, and we shouldn't be framing it as some kind of scam. Second, we don't give terminally ill people "fake treatment" (placebo trials). We give them current standard of care. Giving someone a placebo trial doesn't prove anything that would change clinical practice, because you want to know if the drug works better than what is out there today. Rarely is that standard of care "nothing", and this (bad controls) is actually a primary reason that a lot of drug company trials are rejected by the FDA. If I didn't see the Wall Street Journal editorial board repeating the same garbage in defense of patent medicines, I'd write you off as simply having a sophomoric understanding of how trials work. I'm convinced that someone is driving this absurd narrative.
- A_D_E_P_T 4mo ago
  [flagged]

taeric 4mo ago

This feels off. In medicine, any evidence can also be blinded by confounding factors that are far easier to miss without adding specific controls. Really, in any field this will be the case. Should we demand an RCT before we accept evidence? Of course not. At some point you do have to make a choice on things. And it should be noted that most drugs do have early cutoff criteria if the evidence is strong enough that it is working. It isn't like people are wanting to withhold good treatments from the world. Adding controls and randomizing them, though, has proven to be highly effective at helping progress.

A_D_E_P_T 4mo ago
> "This feels off. In medicine, any evidence can also be blinded by confounding factors that are far easier to miss without adding specific controls. Really, in any field this will be the case." If you have enough data, you can smooth out individual fluctuations due to things like drug interactions, non-compliance, etc. (And indeed you might discover drug interactions!) Observational trials ultimately mirror how drugs are used in the real world. > "Adding controls and randomizing them, though, has proven to be highly effective at helping progress." I would argue just the opposite. Demands for increasingly byzantine trials have ballooned the costs associated with drug development, and have slowed things to a crawl. There's a reason the field's golden age was in the 1940s and 1950s, and it's not just "low hanging fruit." Today nobody in their right mind wants to work in drug development when they could work in tech or even finance.
- taeric 4mo ago
  Right, but you are just relying on a different form of random, there. The whole point of making controls and then building experiments on changing them, is to get more power from fewer observations. No? Again, it is off to think that one is automatically superior to the other. Certainly to the exclusion of the other. And that is what feels off with the framing of the parent post. I am perfectly fine saying you should use both observational and controlled trials. But I think it is also wrong to think you don't have to build experiments to test interventions. This is why you put metrics in your service code. So that you can observe them behave and look for things to change. This is also why you do test cases on your code, so that you can specifically target your change. Now, I fully back the idea that just A/B testing something doesn't automatically mean you learn something true. But neither does observing a strong outcome on uncontrolled data.
  - estearum 4mo ago
    Yeah, GP is basically saying: "Large controlled experiments are costly and can hurt people who opt-in to informed consent. Instead, we should do significantly, significantly larger experiments, with undefined success/failure conditions, and no informed consent." Insane opinion
- RandomLensman 4mo ago
  How do you get enough data? If, for example, you need a lot of people in the sample, that might not be so easy. In the abstract, should it not come done to what is the best experimental design for each case?
- deleted 4mo ago
  [deleted]
- __alexs 4mo ago
  You are simultaneously arguing for a more complex and nuanced testing approach that demands much higher quantities of data as a result, and also against RCTs, which perhaps rightly you've identified as having suffered from the same kind of cost disease as all other health care in the USA. I can't help but feel like you've identified the wrong root cause here.

naasking 4mo ago

> Which simply dooms the terminally ill to fake treatment. I wish people would stop saying this. First, controls aren't necessarily "fake treatment", they are often compared to other standard treatments. Second, the treatment being tested can actually harm the patient more, therefore the people receiving your alleged "fake treatment" can actually come out better off. Which is the "fake treatment" now? I don't disagree with your final point, but mainly with this increasingly pervasive and wrong framing of RCTs.

FeteCommuniste 4mo ago
Yes. Potentially life-saving drugs in RCTs are usually compared to "standard of care," i.e. already-approved treatments, not to placebos.
- qsera 4mo ago
  So if both the standard of care as well as the drug being trials have the same side effects, would they be included in the package insert/drug literature? Because the incidence rate of it would be same for both the groups?

timr 4mo ago

> In medicine, observational evidence is actually better and far more ethical than the RCT. (Which simply dooms the terminally ill to fake treatment.) This is just nonsense. First, everyone in a trial is informed of the situation. It's not "unethical" unless you lie about it. If you participate in a trial, you do so knowing that you might not get the experimental drug. It's a selfless, honorable thing to do, and we shouldn't be framing it as some kind of scam. Second, we don't give terminally ill people "fake treatment" (placebo trials). We give them current standard of care. Giving someone a placebo trial doesn't prove anything that would change clinical practice, because you want to know if the drug works better than what is out there today. Rarely is that standard of care "nothing", and this (bad controls) is actually a primary reason that a lot of drug company trials are rejected by the FDA. If I didn't see the Wall Street Journal editorial board repeating the same garbage in defense of patent medicines, I'd write you off as simply having a sophomoric understanding of how trials work. I'm convinced that someone is driving this absurd narrative.

A_D_E_P_T 4mo ago
[flagged]