5 ms·

Disagree. I change practice on Monday after a single quality trial. Pick up any society guideline, only a small amount of the recommendations rely on meta-analy

by Gatsky 3y ago

Disagree. I change practice on Monday after a single quality trial. Pick up any society guideline, only a small amount of the recommendations rely on meta-analyses. Look at immunotherapy or antibody drug conjugates, revolutionary therapies that arrived one trial at a time.

haldujai 3y ago
For what part, cutting-edge treatments? Sure. But oncology is a very big field… For example, the majority diagnostic testing guidelines are based on meta-analyses. FYI since you asked to pick one, let's take a look at NCCN. Not sure where you're drawing your conclusion that most evidence is from phase III RCTs: "[Q] Quality and quantity of evidence refers to the number and types of clinical trials relevant to a particular intervention. To determine a score, panel members may weigh the depth of the evidence, i.e., the numbers of trials that address this issue and their design. The scale used to measure quality of evidence is: 5 (High quality): Multiple well-designed randomized trials and/or meta-analyses 4 (Good quality): One or more well-designed randomized trials 3 (Average quality): Low quality randomized trial(s) or well-designed non-randomized trial(s) 2 (Low quality): Case reports or extensive clinical experience 1 (Poor quality): Little or no evidence" "The overall quality of the clinical data and evidence that exist within the field of cancer research is highly variable, both within and across cancer types. Large, well designed, randomized controlled trials (RCTs) may provide high-quality clinical evidence in some tumor types and clinical situations. However, much of the clinical evidence available to clinicians is primarily based on data from indirect comparisons among randomized trials, phase II or non-randomized trials, or in many cases, on limited data from multiple smaller trials, retrospective studies, or clinical observations. In some clinical situations, no meaningful clinical data exist and patient care must be based upon clinical experience alone. Thus, in the field of oncology, it becomes critical and necessary (where the evidence landscape remains sparse or suboptimal) to include input from the experience and expertise of cancer specialists and other clinical experts." From an article: "We identified 1124 potential systematic reviews from our survey of the 49 NCCN guidelines for the treatment of cancer by site. Five NCCN guidelines did not cite any systematic reviews." https://www.nccn.org/guidelines/guidelines-process/development-and-update-of-guidelines https://www.nccn.org/guidelines/guidelines-process/developme... https://jamanetwork.com/journals/jamaoncology/fullarticle/2749257 https://jamanetwork.com/journals/jamaoncology/fullarticle/27... https://www.nccn.org/guidelines/guidelines-with-evidence-blocks https://www.nccn.org/guidelines/guidelines-with-evidence-blo...
- Gatsky 3y ago
  Well you discount the most important thing in the first line. 'Cutting-edge' is a funny way of saying 'most effective', as if it were somehow irrelevant. I didn't say most evidence is from phase III RCTs, particularly if you include everything that happens in oncology as the denominator, only that meta-analyses were not that relevant. Most of the critical patient facing interventions have the backing of good quality trials, at least where it is reasonable and possible to do a trial. Also one of your citations is seemingly casting doubt on the value of meta-analyses in oncology, so somewhat confused about your point. That paragraph from NCCN is quite interesting. It is describing medicine in general really, and belies the fact that oncology has probably one of the strongest evidence base across all medical fields. Take for example how many stents cardiologists have inserted long after contradictory evidence was available, or how many pointless back operations have been done, or how many people have sat through fruitless psychoanalysis.
  - haldujai 3y ago
    > Well you discount the most important thing in the first line. 'Cutting-edge' is a funny way of saying 'most effective', as if it were somehow irrelevant. I'm discounting it for this discussion because your argument is: "meta-analyses are somewhat irrelevant" and "Meta-analyses are mostly there for trainees to notch up a paper." which is completely false. Note a single clinical trial is still only considered "good quality" while multiple trials or meta-analyses are considered "high quality". To address this new point you raised, when something has very promising early results we start using it in treatment (e.g. 3rd gen TKIs in adjuvant NSCLC) but until this weekend we had no 5 year OS survival for adjuvant use. It's entirely possible something one thinks is "most effective" is later proven to not be (gen 1-2 TKIs, HIPEC, etc). > That paragraph from NCCN is quite interesting. It is describing medicine in general really, and belies the fact that oncology has probably one of the strongest evidence base across all medical fields. > Take for example how many stents cardiologists have inserted long after contradictory evidence was available, or how many pointless back operations have been done, or how many people have sat through fruitless psychoanalysis. I'm not sure what point you are trying to make by addressing other specialties. The National Comprehensive Cancer Network, comprised of multidisciplinary experts from 33 of the leading cancer centers in the country, is unequivocally the authority in oncology and is incredibly well respected. I'm going to defer to their opinion on the quality of evidence available and the hierarchy of evidence. > Also one of your citations is seemingly casting doubt on the value of meta-analyses in oncology, so somewhat confused about your point. The JAMA article states that the methodology in many studies does not meet NCCN/PRISMA criteria which is a well known, this says nothing about the relative value of good-quality meta-analyses (which are far more common now with the PRISMA update). I'm really not sure why you think systematic reviews are irrelevant, this is a very radical viewpoint that I've seen no evidence of. Good meta-analysis > good RCT. The reality is that good quality studies of both types are uncommon in medicine, but the goal is still to use good SRs.
prashp 3y ago
Maybe we should regulate clinicians more so they don't change their practice as they please and so that healthcare is standardized across the country?
- haldujai 3y ago
  I can't speak to parent's practice but the cancer centers I've worked at follow the NCCN guidelines (apart from patients enrolled in trials, although this is also a NCCN recommendation) and many cases are reviewed in multi-disciplinary conferences to homogenize practice within an institution. Although guidelines are just that (i.e. not mandatory to adhere to) I really doubt an oncologist in US/Canada "practices as they please".
- Gatsky 3y ago
  The president of Medicens sans frontieres, Rowan Gillies, gave a speech once. He had a drawing of a patient and the doctor inside a circle. His comment was 'Other people keep trying to climb into this circle. They can all fuck off.' Excuse his profanity, but I offer this response to you, who knows nothing about what I do, and understands nothing about quality healthcare.
  - prashp 3y ago
    Of course if you ask a clincian if they should be regulated more, they will say no. The fewer stakeholders involved in the relationship between patient and doctor, the more power the doctor has. The truth is there should be others involved, at the very least the government health agencies.
    - Gatsky 3y ago
      Actually doctors would be universally happy to get more help and support to deliver better healthcare. That you don't talk about that is quite telling, or is that what you mean by 'regulation'? Casting the patient doctor relationship in terms of power dynamics is a bogus sociological construct divorced from reality. The true division of power is between those who fund healthcare and those who receive it, I would start there if you think things need to be improved.

by Gatsky 3y ago

haldujai 3y ago

For what part, cutting-edge treatments? Sure. But oncology is a very big field… For example, the majority diagnostic testing guidelines are based on meta-analyses. FYI since you asked to pick one, let's take a look at NCCN. Not sure where you're drawing your conclusion that most evidence is from phase III RCTs: "[Q] Quality and quantity of evidence refers to the number and types of clinical trials relevant to a particular intervention. To determine a score, panel members may weigh the depth of the evidence, i.e., the numbers of trials that address this issue and their design. The scale used to measure quality of evidence is: 5 (High quality): Multiple well-designed randomized trials and/or meta-analyses 4 (Good quality): One or more well-designed randomized trials 3 (Average quality): Low quality randomized trial(s) or well-designed non-randomized trial(s) 2 (Low quality): Case reports or extensive clinical experience 1 (Poor quality): Little or no evidence" "The overall quality of the clinical data and evidence that exist within the field of cancer research is highly variable, both within and across cancer types. Large, well designed, randomized controlled trials (RCTs) may provide high-quality clinical evidence in some tumor types and clinical situations. However, much of the clinical evidence available to clinicians is primarily based on data from indirect comparisons among randomized trials, phase II or non-randomized trials, or in many cases, on limited data from multiple smaller trials, retrospective studies, or clinical observations. In some clinical situations, no meaningful clinical data exist and patient care must be based upon clinical experience alone. Thus, in the field of oncology, it becomes critical and necessary (where the evidence landscape remains sparse or suboptimal) to include input from the experience and expertise of cancer specialists and other clinical experts." From an article: "We identified 1124 potential systematic reviews from our survey of the 49 NCCN guidelines for the treatment of cancer by site. Five NCCN guidelines did not cite any systematic reviews." https://www.nccn.org/guidelines/guidelines-process/development-and-update-of-guidelines https://www.nccn.org/guidelines/guidelines-process/developme... https://jamanetwork.com/journals/jamaoncology/fullarticle/2749257 https://jamanetwork.com/journals/jamaoncology/fullarticle/27... https://www.nccn.org/guidelines/guidelines-with-evidence-blocks https://www.nccn.org/guidelines/guidelines-with-evidence-blo...

Gatsky 3y ago
Well you discount the most important thing in the first line. 'Cutting-edge' is a funny way of saying 'most effective', as if it were somehow irrelevant. I didn't say most evidence is from phase III RCTs, particularly if you include everything that happens in oncology as the denominator, only that meta-analyses were not that relevant. Most of the critical patient facing interventions have the backing of good quality trials, at least where it is reasonable and possible to do a trial. Also one of your citations is seemingly casting doubt on the value of meta-analyses in oncology, so somewhat confused about your point. That paragraph from NCCN is quite interesting. It is describing medicine in general really, and belies the fact that oncology has probably one of the strongest evidence base across all medical fields. Take for example how many stents cardiologists have inserted long after contradictory evidence was available, or how many pointless back operations have been done, or how many people have sat through fruitless psychoanalysis.
- haldujai 3y ago
  > Well you discount the most important thing in the first line. 'Cutting-edge' is a funny way of saying 'most effective', as if it were somehow irrelevant. I'm discounting it for this discussion because your argument is: "meta-analyses are somewhat irrelevant" and "Meta-analyses are mostly there for trainees to notch up a paper." which is completely false. Note a single clinical trial is still only considered "good quality" while multiple trials or meta-analyses are considered "high quality". To address this new point you raised, when something has very promising early results we start using it in treatment (e.g. 3rd gen TKIs in adjuvant NSCLC) but until this weekend we had no 5 year OS survival for adjuvant use. It's entirely possible something one thinks is "most effective" is later proven to not be (gen 1-2 TKIs, HIPEC, etc). > That paragraph from NCCN is quite interesting. It is describing medicine in general really, and belies the fact that oncology has probably one of the strongest evidence base across all medical fields. > Take for example how many stents cardiologists have inserted long after contradictory evidence was available, or how many pointless back operations have been done, or how many people have sat through fruitless psychoanalysis. I'm not sure what point you are trying to make by addressing other specialties. The National Comprehensive Cancer Network, comprised of multidisciplinary experts from 33 of the leading cancer centers in the country, is unequivocally the authority in oncology and is incredibly well respected. I'm going to defer to their opinion on the quality of evidence available and the hierarchy of evidence. > Also one of your citations is seemingly casting doubt on the value of meta-analyses in oncology, so somewhat confused about your point. The JAMA article states that the methodology in many studies does not meet NCCN/PRISMA criteria which is a well known, this says nothing about the relative value of good-quality meta-analyses (which are far more common now with the PRISMA update). I'm really not sure why you think systematic reviews are irrelevant, this is a very radical viewpoint that I've seen no evidence of. Good meta-analysis > good RCT. The reality is that good quality studies of both types are uncommon in medicine, but the goal is still to use good SRs.

prashp 3y ago

Maybe we should regulate clinicians more so they don't change their practice as they please and so that healthcare is standardized across the country?

haldujai 3y ago
I can't speak to parent's practice but the cancer centers I've worked at follow the NCCN guidelines (apart from patients enrolled in trials, although this is also a NCCN recommendation) and many cases are reviewed in multi-disciplinary conferences to homogenize practice within an institution. Although guidelines are just that (i.e. not mandatory to adhere to) I really doubt an oncologist in US/Canada "practices as they please".
Gatsky 3y ago
The president of Medicens sans frontieres, Rowan Gillies, gave a speech once. He had a drawing of a patient and the doctor inside a circle. His comment was 'Other people keep trying to climb into this circle. They can all fuck off.' Excuse his profanity, but I offer this response to you, who knows nothing about what I do, and understands nothing about quality healthcare.
- prashp 3y ago
  Of course if you ask a clincian if they should be regulated more, they will say no. The fewer stakeholders involved in the relationship between patient and doctor, the more power the doctor has. The truth is there should be others involved, at the very least the government health agencies.
  - Gatsky 3y ago
    Actually doctors would be universally happy to get more help and support to deliver better healthcare. That you don't talk about that is quite telling, or is that what you mean by 'regulation'? Casting the patient doctor relationship in terms of power dynamics is a bogus sociological construct divorced from reality. The true division of power is between those who fund healthcare and those who receive it, I would start there if you think things need to be improved.