3 ms·

That's true but this is a clinical trial where the patients are being treated- the patients were selected according to strict criteria, and received other treat

by dekhn 3y ago

That's true but this is a clinical trial where the patients are being treated- the patients were selected according to strict criteria, and received other treatments in cancer centers. It's not really just "coinflips", it's heavily biased coinflips. Maybe I should have put this in a more productive way: I prefer studies that have a large enough n that you can actually measure the failure rate. There's a classic statistics paper where an operation was done on 3 patients and none died- the paper's title is "If nobody dies, is everything OK?" and goes into whether you can reliably conclude causal effects with very small n. Further- let's say the drug caused remission in 50% of the patients, and the other 50% had spontaneous remission (or remissionc aused by other aspects of the treatment). Now it's not 1/2 * 12, it's 1/2 * 6 (or whatever base probability you want to choose). If you compare that probability to the number of people in clinical trials in the US, you would expect that after 30-40 years, you'd see at least one trial with 12 patients with complete remission. (I don't actually doubt the drug is effective. We already know that. And we already know that studies this small really stretch the concepts of significance and causality).

superb_dev 3y ago
I think most people would prefer a study with more participants, but you can’t jump to a large human trial right away
- dekhn 3y ago
  Patient recruitment is hard, but this is a phase II on an already approved drug. Typical Phase II has at least 100 patients.
vikramkr 3y ago
Edit - also, you should go and look up the rates of spontaneous remission for colorectal cancer (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10273780/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10273780/). We're talking about 12 patients with stage 2/3 colorectal cancer, in remission with the treatment alone. What's really cool is that papers tend to have sections where they discuss the statistics so you can use real numbers. Their null hypothesis was a 25% response rate. "This decision rule would result in a type I error rate of 6% if 25% of the patients had an overall response and would provide the study with 84% power if 50% had an overall response. The null hypothesis was established on the basis of a study by Seligmann et al., in which the observed response to chemotherapy among patients with mismatch repair–deficient rectal cancers was 7% (8 of 115 patients)."

by dekhn 3y ago

superb_dev 3y ago

I think most people would prefer a study with more participants, but you can’t jump to a large human trial right away

dekhn 3y ago
Patient recruitment is hard, but this is a phase II on an already approved drug. Typical Phase II has at least 100 patients.

vikramkr 3y ago

Edit - also, you should go and look up the rates of spontaneous remission for colorectal cancer (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10273780/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10273780/). We're talking about 12 patients with stage 2/3 colorectal cancer, in remission with the treatment alone. What's really cool is that papers tend to have sections where they discuss the statistics so you can use real numbers. Their null hypothesis was a 25% response rate. "This decision rule would result in a type I error rate of 6% if 25% of the patients had an overall response and would provide the study with 84% power if 50% had an overall response. The null hypothesis was established on the basis of a study by Seligmann et al., in which the observed response to chemotherapy among patients with mismatch repair–deficient rectal cancers was 7% (8 of 115 patients)."