4 ms·

This sounds good, but there are a lot of practical problems: 1. How do you account for multiple testing? To ensure that the treatment actually works, you need

This sounds good, but there are a lot of practical problems: 1. How do you account for multiple testing? To ensure that the treatment actually works, you need to show a statistically significant difference between control group and experimental group. If you start ten small studies, and expand them based on early results, you are likely to get more false positives (type I error). 2. In proper randomized trials, the statisticians, imaging analysts and sometimes even doctors doctors are blinded from the treatment and outcome until the study has ended. Blinding is especially important for earlier studies, where surrogate endpoints for survival are used (response to treatment in this study) to get quicker results. These measures can be subjective, and thus prone to bias if the studies are not properly blinded. 3. Earlier stage studies are used to estimate the treatment effect, upon which power calculations are based to determine the size of the study population in Phase III clinical trials. This strategy would bias the treatment effect towards a benefit of the drug (by selecting those studies that show a preferential effect). 4. Despite all these objections, note that the results presented are actually interim results of a study that intends to enroll 30 patients. Usually the speed of enrollment of patients is limited by logistics (Finding the right patients to include and getting informed consent from them) rather than the sequential nature of studies. I recommend that you read the 'Statistical analysis' section of the published paper (freely available here: https://www.nejm.org/doi/10.1056/NEJMoa2201445 https://www.nejm.org/doi/10.1056/NEJMoa2201445), and think about how you would write that section with the method you proposed.

BrandoElFollito 4y ago
If we know the statistical evolution of a well known cancer, why not giving the treatment to everyone? If we know that cancer A ends deadly in an an average of 6 months for 90% of the population and if our treatment has wildly different results, why wonder?
- londons_explore 4y ago
  It's important to have a control group to prevent the placebo effect in both the patients and in clinicians. If a patient thinks they are on a treatment, they might actually end up healthier than a patient who has just been told there are no treatment options left for them... Likewise, if a clinician is really hoping the new drug they are developing is successful, their judgement of exactly how quickly a tumour is growing might not be fully impartial. However, there is no need for the control group and treatment group to be 50/50. The control group can also be shared with other studies - although that makes risk budget allocation complex, but not impossible.
  - BrandoElFollito 4y ago
    > If a patient thinks they are on a treatment, they might actually end up healthier than a patient who has just been told there are no treatment options left for them... Isn't the actual aim? to have someone heal? > Likewise, if a clinician is really hoping the new drug they are developing is successful, their judgement of exactly how quickly a tumour is growing might not be fully impartial Well, this is something that is quantifiable I guess (a measurement). And changes must be notable in order to call something a success in that case.
  - carlineng 4y ago
    Clinical trials for cancer treatment rarely use placebos. Instead, the control group is the current standard of care: https://www.cancer.net/research-and-advocacy/clinical-trials/placebos-cancer-clinical-trials https://www.cancer.net/research-and-advocacy/clinical-trials...
- evandijk70 4y ago
  Some reasons to include a control group - Trials tend to be conducted at the best facilities, and they monitor their patients extremely well. Thus, control groups in a trial tend to have a better outcome than the general population. - Patients included in trials tend to be better informed and better motivated to seek treatment. - When trials are run in a specific location, the patient group may be different than the general population (younger/older, less/more exposure to carcinogens, etc.). Note that all of these are in favor of approval for the experimental drug, and combined with financial interest/pressure from pharmaceutical companies to get approval of a drug, leaving out a control group is a recipe for disaster. The hypothetical case you describe where a cancer ends deadly in an average of 6 months for 90% of the population and is suddenly cured by a miracle treatment simply does not happen (see https://www.cancerresearchuk.org/health-professional/cancer-statistics/survival/all-cancers-combined#heading-One https://www.cancerresearchuk.org/health-professional/cancer-... for some statistics), so you will always need a control group.
  - BrandoElFollito 4y ago
    > The hypothetical case you describe where a cancer ends deadly in an average of 6 months for 90% of the population and is suddenly cured by a miracle treatment simply does not happen That was an exaggeration (in line with the title of the article) - what I meant are cases where we historically have a good knowledge of the trajectory of an illness and this could be used as the control group.
londons_explore 4y ago
> Multiple testing I imagine that the 'risk budget' would be divided up somehow. So a company can decide to use its risk budget on 10 small studies (potentially of slightly different treatments) or one large study. They can mix-n-match and end one study early to reallocate the risk to another study to accelerate it, etc. Governments could allocate risk budgets to bio-tech companies or divide it per disease etc. The smaller initial risk budget of each smaller study decreases the chance of type I error, and the two effects could be designed to cancel. > blinding This approach doesn't prevent blinding. You're right that it makes it more complex. You could imagine a computer system (or independant human) that takes as input all the study results (ie. observations seen so far per patient), unblinds them, does predetermined statistical analysis, and outputs a number of new patients that may join the study today. That number could have a small amount of noise added to prevent "patient number 27 got a headache, and our patient limit fell, therefore patient 27 must be part of the treatment group". > speed of enrollment of patients is limited by logistics For the diseases that have the most human impact, this isn't the case. 50,000 people per day die of heart disease. I don't think any study will struggle to find a few hundred willing participants for a trial. For a disease with very few patients, the proposed scheme doesn't help, but also doesn't hurt.
- mcbain 4y ago
  Regarding blinding: I was part of a clinical trial (phase III, not the drug or cancer this article relates to). It couldn't be blinded - the side effects of the trial drug were too significant compared to just the standard of care which was the control. Also it would have been obvious during infusion, etc. There are many different ways to conduct research like this, of course there is no single perfect methodology.
- evandijk70 4y ago
  > Risk budget How does a smaller risk budget reduce the chance of type I error? Also, a lot of care would have to be taken that all smaller studies are pre-registered as well as their outcomes. > Unblinding In practice, things happen that were not foreseen in the predefined analysis plan. Still, that solution may work provided that the statistical issues were solved correctly. > speed of enrolment Patient groups are often very selected, and almost all studies struggle to find sufficient participants. This leads to delay for many studies, and 1 in 10 studies are even prematurely stopped solely because they cannot find enough participants [1]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4843106/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4843106/

BrandoElFollito 4y ago

If we know the statistical evolution of a well known cancer, why not giving the treatment to everyone? If we know that cancer A ends deadly in an an average of 6 months for 90% of the population and if our treatment has wildly different results, why wonder?

londons_explore 4y ago
It's important to have a control group to prevent the placebo effect in both the patients and in clinicians. If a patient thinks they are on a treatment, they might actually end up healthier than a patient who has just been told there are no treatment options left for them... Likewise, if a clinician is really hoping the new drug they are developing is successful, their judgement of exactly how quickly a tumour is growing might not be fully impartial. However, there is no need for the control group and treatment group to be 50/50. The control group can also be shared with other studies - although that makes risk budget allocation complex, but not impossible.
- BrandoElFollito 4y ago
  > If a patient thinks they are on a treatment, they might actually end up healthier than a patient who has just been told there are no treatment options left for them... Isn't the actual aim? to have someone heal? > Likewise, if a clinician is really hoping the new drug they are developing is successful, their judgement of exactly how quickly a tumour is growing might not be fully impartial Well, this is something that is quantifiable I guess (a measurement). And changes must be notable in order to call something a success in that case.
- carlineng 4y ago
  Clinical trials for cancer treatment rarely use placebos. Instead, the control group is the current standard of care: https://www.cancer.net/research-and-advocacy/clinical-trials/placebos-cancer-clinical-trials https://www.cancer.net/research-and-advocacy/clinical-trials...
evandijk70 4y ago
Some reasons to include a control group - Trials tend to be conducted at the best facilities, and they monitor their patients extremely well. Thus, control groups in a trial tend to have a better outcome than the general population. - Patients included in trials tend to be better informed and better motivated to seek treatment. - When trials are run in a specific location, the patient group may be different than the general population (younger/older, less/more exposure to carcinogens, etc.). Note that all of these are in favor of approval for the experimental drug, and combined with financial interest/pressure from pharmaceutical companies to get approval of a drug, leaving out a control group is a recipe for disaster. The hypothetical case you describe where a cancer ends deadly in an average of 6 months for 90% of the population and is suddenly cured by a miracle treatment simply does not happen (see https://www.cancerresearchuk.org/health-professional/cancer-statistics/survival/all-cancers-combined#heading-One https://www.cancerresearchuk.org/health-professional/cancer-... for some statistics), so you will always need a control group.
- BrandoElFollito 4y ago
  > The hypothetical case you describe where a cancer ends deadly in an average of 6 months for 90% of the population and is suddenly cured by a miracle treatment simply does not happen That was an exaggeration (in line with the title of the article) - what I meant are cases where we historically have a good knowledge of the trajectory of an illness and this could be used as the control group.

londons_explore 4y ago

> Multiple testing I imagine that the 'risk budget' would be divided up somehow. So a company can decide to use its risk budget on 10 small studies (potentially of slightly different treatments) or one large study. They can mix-n-match and end one study early to reallocate the risk to another study to accelerate it, etc. Governments could allocate risk budgets to bio-tech companies or divide it per disease etc. The smaller initial risk budget of each smaller study decreases the chance of type I error, and the two effects could be designed to cancel. > blinding This approach doesn't prevent blinding. You're right that it makes it more complex. You could imagine a computer system (or independant human) that takes as input all the study results (ie. observations seen so far per patient), unblinds them, does predetermined statistical analysis, and outputs a number of new patients that may join the study today. That number could have a small amount of noise added to prevent "patient number 27 got a headache, and our patient limit fell, therefore patient 27 must be part of the treatment group". > speed of enrollment of patients is limited by logistics For the diseases that have the most human impact, this isn't the case. 50,000 people per day die of heart disease. I don't think any study will struggle to find a few hundred willing participants for a trial. For a disease with very few patients, the proposed scheme doesn't help, but also doesn't hurt.

mcbain 4y ago
Regarding blinding: I was part of a clinical trial (phase III, not the drug or cancer this article relates to). It couldn't be blinded - the side effects of the trial drug were too significant compared to just the standard of care which was the control. Also it would have been obvious during infusion, etc. There are many different ways to conduct research like this, of course there is no single perfect methodology.
evandijk70 4y ago
> Risk budget How does a smaller risk budget reduce the chance of type I error? Also, a lot of care would have to be taken that all smaller studies are pre-registered as well as their outcomes. > Unblinding In practice, things happen that were not foreseen in the predefined analysis plan. Still, that solution may work provided that the statistical issues were solved correctly. > speed of enrolment Patient groups are often very selected, and almost all studies struggle to find sufficient participants. This leads to delay for many studies, and 1 in 10 studies are even prematurely stopped solely because they cannot find enough participants [1]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4843106/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4843106/