5 ms·

Some napkin math: Given rectal cancer's rate of survival of 67%, and the small size of the study (18 people), you should see similar results due to random chanc

Some napkin math: Given rectal cancer's rate of survival of 67%, and the small size of the study (18 people), you should see similar results due to random chance every 1350th study. A cursory search on clinicaltrials.gov and I can find 7131 cancer studies started in 2021 alone. It's therefore not unreasonable for this one to be just a random fluke.

rossdavidh 4y ago
Your basic result stands, but "survival" and "remission" don't necessarily equate. But I agree with your basic point.
yakak 4y ago
If I understand the article correctly they were excluding patients enrolled in chemo and radiation? If that's what they mean the survival rate wouldn't be 67% so this would imply a 1349 in 1350 chance the treatment is better than the average treatment?
inglor_cz 4y ago
This napkin math ignores some very significant circumstances. People who suffer from rectal cancer usually undergo surgery to remove the primary tumor. But those trial patients weren't operated on, their treatment was non-invasive. How many rectal cancer sufferers who never undergo a surgery survive? I would bet that it is a lot less than 67 per cent.
- cowmoo728 4y ago
  And this was people with locally advanced rectal cancer. Typically this means the tumor has grown to a considerable size and is already causing symptoms severe enough for people to go to a specialist. I am not a doctor but my understanding is that a placebo in this case would have a 5 year survival rate that is pretty close to 0%.
raindear 4y ago
They did not just survive. They had no traces of cancer. That chance is much much lower.
goodpoint 4y ago
Full remission without surgery? This looks much more like tossing a dice 18 times and getting a 6 every single time!
- fnordpiglet 4y ago
  With 1000 sided die perhaps
X6S1x6Okd1st 4y ago
Pretty sure the base rate you'd want to compare against is either placebo given -> remission or simply spontaneous remission. It appears that spontaneous remission is really quite rare. > [sponanteous remission of cancer] incidence is roughly one in every 60 000–100 000 cancer patients, but the true figure is unknown (2). Spontaneous regression of colon cancer seems to be particularly rare https://academic.oup.com/jjco/article/45/1/111/888056 https://academic.oup.com/jjco/article/45/1/111/888056 So using 0.0016% and 12 patients (which is what the paper the NYT actually links) For a ~50% chance of seeing one trial with 12 patients have complete remission you'd expect to see ~43k trails. (1 - 0.000016)^43000 = 0.502577 I wouldn't discount this study based off of those numbers.
- Aransentin 4y ago
  If spontaneous remission is super rare it does change the conclusion, yeah. (Tangentially I think your stats are a little messed up there - you calculated the expected number of people out of 43000 surviving, not the trials returning a false positive. Assuming 0.0016% that chance is astronomically small, so your argument is stronger.)
  - X6S1x6Okd1st 4y ago
    whoops good point!
fluidcruft 4y ago
7131 rectal cancer studies or just cancer studies in general?
hirako2000 4y ago
If i put a single coin to the vending machine and get 7 cans instead of 0.98, i surely will try a few times more before reaching the conclusion my coins are magical beans. Visibly medical research jump so quick to conclusion it's to the millions of news reader to swallow the clickbaits.
iskander 4y ago
There should be some kind of award for these kinds of "well actually" comments on HN that lack any kind of intuition for the domain. Cartoon montage: "By my calculations..." followed by driving a car off a bridge. Edit: as someone who works in cancer research, I can tell you that your prior for 18/18 locally advanced colorectal cancer patients achieve CRs without surgery should be ~0.
rad88 4y ago
This analysis is mindless and inappropriate. If you care about this at all do yourself a favor and just read the study. https://www.nejm.org/doi/full/10.1056/NEJMoa2201445 https://www.nejm.org/doi/full/10.1056/NEJMoa2201445 Otherwise redo your napkin math and cursory search to answer, specifically, whether all these cancers disappearing within weeks of dostarlimab treatment could be a fluke. And do not compare this to "rectal cancer's rate of survival", which is irrelevant and a completely different set of (parametrized) statistics, and also do not compare it to the total number of "cancer studies", which was an arbitrary choice and yielded this meaningless conclusion. Even if this kind of analysis was useful, why did you compare against the number of cancer studies, rather than rectal cancer (1910), or dostarlimab (41), or studies with the same staging and genetic pathology? It's meaningless. I don't believe you're qualified to tell anyone about the significance of this study, and much less dismiss it as a fluke.
- fnordpiglet 4y ago
  I bet he’s good at programming though.
  - hahaxdxd123 4y ago
    This is one of my favourite HN threads LMAO.
    - system2 4y ago
      Turned into Reddit's armchair experts' discussion.
    - toolslive 4y ago
      > LMAO Pun intended?
  - aaaaaaaaaaab 4y ago
    Most definitely!
- Aransentin 4y ago
wonnage 4y ago
Other commenters have done a fine job expressing why this napkin math is silly and perhaps in isolated cases this is a good thing, OP probably learned something here But in aggregate these sorts of comments are annoying as hell on every medical article posted to HN. Now you have to hope a sufficient mass of well-informed commenters is here to rebut them. In the best-case, these comments are simply misguided, but in the worst case it becomes a watering hole for all the antivaxxers and conspiracy theorists on this site to gather

rossdavidh 4y ago

Your basic result stands, but "survival" and "remission" don't necessarily equate. But I agree with your basic point.

yakak 4y ago

If I understand the article correctly they were excluding patients enrolled in chemo and radiation? If that's what they mean the survival rate wouldn't be 67% so this would imply a 1349 in 1350 chance the treatment is better than the average treatment?

inglor_cz 4y ago

This napkin math ignores some very significant circumstances. People who suffer from rectal cancer usually undergo surgery to remove the primary tumor. But those trial patients weren't operated on, their treatment was non-invasive. How many rectal cancer sufferers who never undergo a surgery survive? I would bet that it is a lot less than 67 per cent.

cowmoo728 4y ago
And this was people with locally advanced rectal cancer. Typically this means the tumor has grown to a considerable size and is already causing symptoms severe enough for people to go to a specialist. I am not a doctor but my understanding is that a placebo in this case would have a 5 year survival rate that is pretty close to 0%.

raindear 4y ago

They did not just survive. They had no traces of cancer. That chance is much much lower.

goodpoint 4y ago

Full remission without surgery? This looks much more like tossing a dice 18 times and getting a 6 every single time!

fnordpiglet 4y ago
With 1000 sided die perhaps

X6S1x6Okd1st 4y ago

Pretty sure the base rate you'd want to compare against is either placebo given -> remission or simply spontaneous remission. It appears that spontaneous remission is really quite rare. > [sponanteous remission of cancer] incidence is roughly one in every 60 000–100 000 cancer patients, but the true figure is unknown (2). Spontaneous regression of colon cancer seems to be particularly rare https://academic.oup.com/jjco/article/45/1/111/888056 https://academic.oup.com/jjco/article/45/1/111/888056 So using 0.0016% and 12 patients (which is what the paper the NYT actually links) For a ~50% chance of seeing one trial with 12 patients have complete remission you'd expect to see ~43k trails. (1 - 0.000016)^43000 = 0.502577 I wouldn't discount this study based off of those numbers.

Aransentin 4y ago
If spontaneous remission is super rare it does change the conclusion, yeah. (Tangentially I think your stats are a little messed up there - you calculated the expected number of people out of 43000 surviving, not the trials returning a false positive. Assuming 0.0016% that chance is astronomically small, so your argument is stronger.)
- X6S1x6Okd1st 4y ago
  whoops good point!

fluidcruft 4y ago

7131 rectal cancer studies or just cancer studies in general?

hirako2000 4y ago

If i put a single coin to the vending machine and get 7 cans instead of 0.98, i surely will try a few times more before reaching the conclusion my coins are magical beans. Visibly medical research jump so quick to conclusion it's to the millions of news reader to swallow the clickbaits.

iskander 4y ago

There should be some kind of award for these kinds of "well actually" comments on HN that lack any kind of intuition for the domain. Cartoon montage: "By my calculations..." followed by driving a car off a bridge. Edit: as someone who works in cancer research, I can tell you that your prior for 18/18 locally advanced colorectal cancer patients achieve CRs without surgery should be ~0.

rad88 4y ago

This analysis is mindless and inappropriate. If you care about this at all do yourself a favor and just read the study. https://www.nejm.org/doi/full/10.1056/NEJMoa2201445 https://www.nejm.org/doi/full/10.1056/NEJMoa2201445 Otherwise redo your napkin math and cursory search to answer, specifically, whether all these cancers disappearing within weeks of dostarlimab treatment could be a fluke. And do not compare this to "rectal cancer's rate of survival", which is irrelevant and a completely different set of (parametrized) statistics, and also do not compare it to the total number of "cancer studies", which was an arbitrary choice and yielded this meaningless conclusion. Even if this kind of analysis was useful, why did you compare against the number of cancer studies, rather than rectal cancer (1910), or dostarlimab (41), or studies with the same staging and genetic pathology? It's meaningless. I don't believe you're qualified to tell anyone about the significance of this study, and much less dismiss it as a fluke.

fnordpiglet 4y ago
I bet he’s good at programming though.
- hahaxdxd123 4y ago
  This is one of my favourite HN threads LMAO.
  - system2 4y ago
    Turned into Reddit's armchair experts' discussion.
  - toolslive 4y ago
    > LMAO Pun intended?
- aaaaaaaaaaab 4y ago
  Most definitely!
Aransentin 4y ago

wonnage 4y ago

Other commenters have done a fine job expressing why this napkin math is silly and perhaps in isolated cases this is a good thing, OP probably learned something here But in aggregate these sorts of comments are annoying as hell on every medical article posted to HN. Now you have to hope a sufficient mass of well-informed commenters is here to rebut them. In the best-case, these comments are simply misguided, but in the worst case it becomes a watering hole for all the antivaxxers and conspiracy theorists on this site to gather