3 ms·
For cases like this, exponentially scaled studies should be allowed. Current practice is to enroll 14 participants (and some control participants) and wait man
by londons_explore 4y ago
For cases like this, exponentially scaled studies should be allowed.
Current practice is to enroll 14 participants (and some control participants) and wait many months for the results to roll in, which, if the treatment is effective lets people not on the treatment die unnecessarily.
A better approach is to have a rolling approach. Ie. Every time a potential participant is found, they are allowed to join the study if the results from all other participants show that it is statistically beneficial, minus a risk budget.
So if a treatment does really well like this, then it will quickly expand to a large number of participants. There is no need to wait for results - as soon as the current cohort of participants survives a few extra days, or as soon as someone in the control group dies, you can enroll a few more participants.
And if a treatment is marginal, then the group size will only expand very slowly, or not at all, as more and more data is collected from existing participants. If the data starts to show a detrimental effect of the same size or larger than the risk budget, then the study can take on no new participants, and if the collected data is not enough to approve the treatment then the result is a failure.
- londons_explore 4y agoThis also opens the door to eliminating stage 1/2/3 trials. Let the study leadership decide how to spread their risk budget, and to overlap them if they choose to.
- refurb 4y agoThey already do this, especially for cancer. Phase 3 trials aren't always needed. Phase I/IIb are common.
- lrem 4y agoI imagine the test protocols are heavily regulated. Getting what you describe right is within reach of experiment design... But updating regulations in a mathematically correct way is a whole new challenge. And given the current state of globalisation, an interested party would need to convince multiple government bodies, using multiple languages. Which means we'd need an interested party with deep pockets and good motivation.
- evandijk70 4y agoThis sounds good, but there are a lot of practical problems: 1. How do you account for multiple testing? To ensure that the treatment actually works, you need to show a statistically significant difference between control group and experimental group. If you start ten small studies, and expand them based on early results, you are likely to get more false positives (type I error). 2. In proper randomized trials, the statisticians, imaging analysts and sometimes even doctors doctors are blinded from the treatment and outcome until the study has ended. Blinding is especially important for earlier studies, where surrogate endpoints for survival are used (response to treatment in this study) to get quicker results. These measures can be subjective, and thus prone to bias if the studies are not properly blinded. 3. Earlier stage studies are used to estimate the treatment effect, upon which power calculations are based to determine the size of the study population in Phase III clinical trials. This strategy would bias the treatment effect towards a benefit of the drug (by selecting those studies that show a preferential effect). 4. Despite all these objections, note that the results presented are actually interim results of a study that intends to enroll 30 patients. Usually the speed of enrollment of patients is limited by logistics (Finding the right patients to include and getting informed consent from them) rather than the sequential nature of studies. I recommend that you read the 'Statistical analysis' section of the published paper (freely available here: https://www.nejm.org/doi/10.1056/NEJMoa2201445 https://www.nejm.org/doi/10.1056/NEJMoa2201445), and think about how you would write that section with the method you proposed.
- BrandoElFollito 4y agoIf we know the statistical evolution of a well known cancer, why not giving the treatment to everyone? If we know that cancer A ends deadly in an an average of 6 months for 90% of the population and if our treatment has wildly different results, why wonder?
- londons_explore 4y agoIt's important to have a control group to prevent the placebo effect in both the patients and in clinicians. If a patient thinks they are on a treatment, they might actually end up healthier than a patient who has just been told there are no treatment options left for them... Likewise, if a clinician is really hoping the new drug they are developing is successful, their judgement of exactly how quickly a tumour is growing might not be fully impartial. However, there is no need for the control group and treatment group to be 50/50. The control group can also be shared with other studies - although that makes risk budget allocation complex, but not impossible.
- pintxo 4y agoStudies will actually be stopped short if results are so good as it would be unethical to continue giving people placebos. It happens, but rarely.
- throwayyy479087 4y agoI believe this happened with Gleevec
- refurb 4y agoThey already do this. It’s not unusual to add “treatment arms” to a study. However, that still wouldn’t be enough. This was a single arm uncontrolled study. No control arm, every patient knew they were getting treatment as did the doctors. And since you need to pre-define your study design and endpoints, you’d basically need a more rigorously designed trial to actually get approval. But all this is intentional. You start with quick, cheap studies to see if it’s worthwhile spending more.
- mchusma 4y agoThe FDA also currently has basically 2 statuses: "banned" and "covered by insurance". Making an intermediary step which is "not banned not covered", would allow things to progress faster. I'm ok with reporting requirements allowing for this phase to act like a large unblinded test too.