8 ms·

RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses lik

RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses like cancer: https://jakeseliger.com/2024/01/29/the-dead-and-dying-at-the-gates-of-oncology-clinical-trials/ https://jakeseliger.com/2024/01/29/the-dead-and-dying-at-the.... We have Kaplan-Meier curves for fatal diagnoses. We know what happens (the tumors grow and metastasize. One doesn't need elaborate phase 3 RCTs to figure out if there's a good shot that a treatment is working; one can see it in tumor response and comparison to known KMCs. The existing system raises costs and causes people to die while waiting a decade or more for exciting treatments: https://atelfo.github.io/2023/12/23/biopharma-from-janssen-to-today.html https://atelfo.github.io/2023/12/23/biopharma-from-janssen-t... Moderna's mRNA-4157 is a current example of this: https://jakeseliger.com/2024/04/12/moderna-mrna-4157-v90-news-for-head-and-neck-cancer-patients-like-me/ https://jakeseliger.com/2024/04/12/moderna-mrna-4157-v90-new..., although it may be held up by lack of manufacturing capacity as well.

david_shi 2y ago
100% agreed. "RCT's are the gold standard" doesn't make them gospel.
Aurornis 2y ago
> RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses like cancer: RCTs for mental health conditions are a completely different situation. The short-term placebo response rate for cancers is not high (obviously) though the influence of unblinded trial operators making subjective analyses can be a problem. Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial. The magnitude of the placebo response is almost hard to believe in certain studies. The placebo effect can be a problem for approving new drugs as some times the placebo group improved so much that there isn’t much room left for the active drug to improve beyond that. This is a problem of study design and rating systems that is difficult to solve. Unfortunately, some study operators use this fact to their advantage by omitting placebo group. Without a placebo group, it’s not obvious that the drug is actually doing anything better than placebo, of course.
- BurningFrog 2y ago
  > * Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial* Probably how faith healing works.
  - vitiral 2y ago
    If someone could explain how "the placebo effect" is not just "scientifically proven faith healing" I'd love to hear it.
    - otherme123 2y ago
      Placebos doesn't heal. They just seem to relief some symptoms typically self reported, like pain, but the underlying cause is still there.
      - Xymist 2y ago
        The tricky thing about that (which isn't false, per se) in the context of mental health is that "relieve some self reported symptoms" can actually be sufficient treatment. As with many sorts of pain, if the patient feels better, _they are better_ in a meaningful sense. Whether it's "real" is sort of beside the point, especially if the problem is that they are (for example) too miserable to do normal life things that would stop them being miserable and the placebo is sufficient for them to feel as if perhaps they could.
        lukas099 2y ago
        Also, relief of emotional discomfort can help the patient adopt more behaviors that are associated with positive changes in mental health, such as exercise and pursuing social engagements.
        Uehreka 2y ago
        Not necessarily. The placebo effect is the kind of thing where you might have trouble at first telling if your symptoms are improving or if you’re just having a “good day” or an “easy week”, and that confusion can even last a month or two during which you’re over-observing your internal state and feeling hopeful that “maybe this is what getting better feels like”. But in the long term you often figure it out. I had a placebo effect recently when switching ADHD medication to get around the shortages. For a couple months I thought there was a chance my new meds might actually be better, they definitely felt different (and still do). But six months in it’s clear to me that I’m struggling with productivity more than I was before I switched (though less than when I was off meds). I’m just one guy, but I’d guess this is why doctors don’t just prescribe placebos all the time as actual therapies (well, that and they’d lose credibility which would then destroy any remaining placebo effect).
refurb 2y ago
> One doesn't need elaborate phase 3 RCTs to figure out if there's a good shot that a treatment is working....The existing system raises costs and causes people to die while waiting a decade or more for exciting treatments The FDA often approves cancer drugs without a phase 3 randomized trial. In fact, most new cancer drugs are approved without a phase 3 trial. Just taking a random cancer drug from this list: https://www.fda.gov/drugs/novel-drug-approvals-fda/novel-drug-approvals-2024 https://www.fda.gov/drugs/novel-drug-approvals-fda/novel-dru... "The efficacy of IMDELLTRA was evaluated in Study DeLLphi-301 [NCT05060016], an open-label, multicenter, multi-cohort clinical trial....A total of 99 patients received IMDELLTRA..." This is a new small cell lung cancer drug approved via a phase 2 study that didn't have a control arm and wasn't blinded. This is pretty typical. > one can see it in tumor response and comparison to known KMCs. Anything measured by a human can be biased by knowledge that a patient received a treatment, including tumor response (often blobs on a screen from a FDG PET/CT scan.) RCTs are the gold standard. We don't need to start chipping away at the rigorous standards we have in place to accurately measure the value that a medicine offers. What we can do - and are doing right now - is do a risk-benefit analysis and allow drugs to be approved with a weaker set of data so that patients with a life-threatening illness can get access earlier.
tsimionescu 2y ago
One thing that doesn't often get appreciated in such discussions is that there are a lot of drugs that seem promising at first, but fizzle out in larger trials. If drug companies had a known pathway to go from positive initial results to very expedited approval, even for limited cases, you can be absolutely sure that they would game the hell out of this system to sell "miracle drugs" to desperate dying patients who will pay anything for a chance. While it's sad and horrible to know that a cure for your condition may already exist and be just out of reach, and I can imagine the despair at that, I'm not convinced the alternative is all that more appealing. I would also note that it's certainly not, by any stretch, the worse injustice in the medical system. For every one patient with a terrible cancer that might have survived if allowed access to an experimental treatment, there are millions of people dying of easily treatable diseases for which we have had a treatment for the last hundred years, but who can't afford it. The existence of a cure for your condition that you just can't access for whatever reason is a reality of our system. Caution in introducing new drugs is actually one of the more rational reasons, that one needs to try to come to terms with.
- collyw 2y ago
  Got to remember that there is a great financial incentive for trials to come out positive. Who pays for clinical trials?
eviks 2y ago
There is way too little obsession with them given how much of research isn't using/reusing this method And specifically regarding cancer we also know a lot of very extensive drugs fail at reducing mortality (specifically, as far as I remember, tumor reduction may have no connection to mortality for some cancers, so we don't really "know what happens" without factual data)
moomin 2y ago
Memorably, someone in the UK recently dismissed over 100 studies because they didn’t perform double blind RCT. Which is hilarious when you consider the studies were about gender affirming/reassigning treatments.
mharig 2y ago
Furthermore, RCTs just identify the substances that have positive effect on the largest subgroup of the studied group (which 'coincidently' is what the Pharma Mafia is most interested in). But we are all individuals, differing in genetic, epigenetic, foreign organisms, metabolism. The near only value the RCTs have to an individuum is to determine the substance(s) which most probably have an effect.

david_shi 2y ago

100% agreed. "RCT's are the gold standard" doesn't make them gospel.

Aurornis 2y ago

> RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses like cancer: RCTs for mental health conditions are a completely different situation. The short-term placebo response rate for cancers is not high (obviously) though the influence of unblinded trial operators making subjective analyses can be a problem. Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial. The magnitude of the placebo response is almost hard to believe in certain studies. The placebo effect can be a problem for approving new drugs as some times the placebo group improved so much that there isn’t much room left for the active drug to improve beyond that. This is a problem of study design and rating systems that is difficult to solve. Unfortunately, some study operators use this fact to their advantage by omitting placebo group. Without a placebo group, it’s not obvious that the drug is actually doing anything better than placebo, of course.

BurningFrog 2y ago
> * Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial* Probably how faith healing works.
- vitiral 2y ago
  If someone could explain how "the placebo effect" is not just "scientifically proven faith healing" I'd love to hear it.
  - otherme123 2y ago
    Placebos doesn't heal. They just seem to relief some symptoms typically self reported, like pain, but the underlying cause is still there.
    - Xymist 2y ago
      The tricky thing about that (which isn't false, per se) in the context of mental health is that "relieve some self reported symptoms" can actually be sufficient treatment. As with many sorts of pain, if the patient feels better, _they are better_ in a meaningful sense. Whether it's "real" is sort of beside the point, especially if the problem is that they are (for example) too miserable to do normal life things that would stop them being miserable and the placebo is sufficient for them to feel as if perhaps they could.
      - lukas099 2y ago
        Also, relief of emotional discomfort can help the patient adopt more behaviors that are associated with positive changes in mental health, such as exercise and pursuing social engagements.
      - Uehreka 2y ago
        Not necessarily. The placebo effect is the kind of thing where you might have trouble at first telling if your symptoms are improving or if you’re just having a “good day” or an “easy week”, and that confusion can even last a month or two during which you’re over-observing your internal state and feeling hopeful that “maybe this is what getting better feels like”. But in the long term you often figure it out. I had a placebo effect recently when switching ADHD medication to get around the shortages. For a couple months I thought there was a chance my new meds might actually be better, they definitely felt different (and still do). But six months in it’s clear to me that I’m struggling with productivity more than I was before I switched (though less than when I was off meds). I’m just one guy, but I’d guess this is why doctors don’t just prescribe placebos all the time as actual therapies (well, that and they’d lose credibility which would then destroy any remaining placebo effect).

refurb 2y ago

> One doesn't need elaborate phase 3 RCTs to figure out if there's a good shot that a treatment is working....The existing system raises costs and causes people to die while waiting a decade or more for exciting treatments The FDA often approves cancer drugs without a phase 3 randomized trial. In fact, most new cancer drugs are approved without a phase 3 trial. Just taking a random cancer drug from this list: https://www.fda.gov/drugs/novel-drug-approvals-fda/novel-drug-approvals-2024 https://www.fda.gov/drugs/novel-drug-approvals-fda/novel-dru... "The efficacy of IMDELLTRA was evaluated in Study DeLLphi-301 [NCT05060016], an open-label, multicenter, multi-cohort clinical trial....A total of 99 patients received IMDELLTRA..." This is a new small cell lung cancer drug approved via a phase 2 study that didn't have a control arm and wasn't blinded. This is pretty typical. > one can see it in tumor response and comparison to known KMCs. Anything measured by a human can be biased by knowledge that a patient received a treatment, including tumor response (often blobs on a screen from a FDG PET/CT scan.) RCTs are the gold standard. We don't need to start chipping away at the rigorous standards we have in place to accurately measure the value that a medicine offers. What we can do - and are doing right now - is do a risk-benefit analysis and allow drugs to be approved with a weaker set of data so that patients with a life-threatening illness can get access earlier.

tsimionescu 2y ago

One thing that doesn't often get appreciated in such discussions is that there are a lot of drugs that seem promising at first, but fizzle out in larger trials. If drug companies had a known pathway to go from positive initial results to very expedited approval, even for limited cases, you can be absolutely sure that they would game the hell out of this system to sell "miracle drugs" to desperate dying patients who will pay anything for a chance. While it's sad and horrible to know that a cure for your condition may already exist and be just out of reach, and I can imagine the despair at that, I'm not convinced the alternative is all that more appealing. I would also note that it's certainly not, by any stretch, the worse injustice in the medical system. For every one patient with a terrible cancer that might have survived if allowed access to an experimental treatment, there are millions of people dying of easily treatable diseases for which we have had a treatment for the last hundred years, but who can't afford it. The existence of a cure for your condition that you just can't access for whatever reason is a reality of our system. Caution in introducing new drugs is actually one of the more rational reasons, that one needs to try to come to terms with.

collyw 2y ago
Got to remember that there is a great financial incentive for trials to come out positive. Who pays for clinical trials?

eviks 2y ago

There is way too little obsession with them given how much of research isn't using/reusing this method And specifically regarding cancer we also know a lot of very extensive drugs fail at reducing mortality (specifically, as far as I remember, tumor reduction may have no connection to mortality for some cancers, so we don't really "know what happens" without factual data)

moomin 2y ago

Memorably, someone in the UK recently dismissed over 100 studies because they didn’t perform double blind RCT. Which is hilarious when you consider the studies were about gender affirming/reassigning treatments.

mharig 2y ago

Furthermore, RCTs just identify the substances that have positive effect on the largest subgroup of the studied group (which 'coincidently' is what the Pharma Mafia is most interested in). But we are all individuals, differing in genetic, epigenetic, foreign organisms, metabolism. The near only value the RCTs have to an individuum is to determine the substance(s) which most probably have an effect.