5 ms·

"When all 14 studies were pooled, no statistically significant results emerged. However, when the studies were divided according to whether vitamin D3 was taken

• mholm 3y ago
  That's what's confusing me: 105,000 participants in a study should have resulted in thousands of cancer deaths, a 12% drop should be pretty noticeable even with 10% of the data.
  • kelseyfrog 3y ago
    How exactly are you getting "should have resulted in thousands of cancer deaths"? A population of 100,000 should expect to see 144.1 deaths[1]. If you evenly divide the study group into two 52,500-sized groups(treatment + control), each group should expect to see 72 deaths. A 12% drop in one of the groups is 8.6 deaths. As a colleague would say, that's very few clams from which to make a chowder. Perhaps the researchers are using more advanced tools than I am, but even assuming a 12% difference between treatment and control does not result in a chi-squared test with any reasonable significance. ie: the null-hypothesis(there is no difference between groups) cannot be rejected. 1. https://www.cdc.gov/cancer/dcpc/research/update-on-cancer-deaths/index.htm https://www.cdc.gov/cancer/dcpc/research/update-on-cancer-de...
    • mholm 3y ago
      144.1 deaths per year. Presumably these studies lasted multiple years. Though I'll admit this was less than I expected, which would explain the differing results.
      • kelseyfrog 3y ago
        Thank you! Yes, the time frame is important, d'uh lol.
• cypress66 3y ago
  https://xkcd.com/882/ https://xkcd.com/882/
• amatecha 3y ago
  Yeah, in addition to the XKCD comic it also reminds me of "Spurious Correlations" lol https://tylervigen.com/spurious-correlations https://tylervigen.com/spurious-correlations
• CodeWriter23 3y ago
  D3 (animal-based) > D2 (plant-based) > D1 (synthetic), at least in humans if you read the literature, that is. The fishy thing is how actual scientists do whole studies using only D1 and then draw conclusions.
  • Filligree 3y ago
    My takeaway is to keep eating fish.
    • AlecSchueler 3y ago
      Overfishing is serious enough at this point that I'd rather take a small health risk than contribute to it.
      • deleted 3y ago
        [deleted]
      • bt4u 3y ago
        [dead]
      • onlyrealcuzzo 3y ago
        Does aquaponic tilapia not have D3?
      • candiddevmike 3y ago
        Not sure why you are being downvoted. Salmon (mentioned in other siblings) is a perfect example of over fishing, and the farmed stuff tends to be of terrible quality and fed absolute garbage/treated terribly.
        • tablarasa 3y ago
          Salmon is found in high-latitude systems around the globe in both hemispheres. Overfishing is something that occurs to populations, not species. Most of the salmon populations that I am familiar with are not over-fished, but have suffered from spawning habitat loss due to dams and logging. Those are major issues and they continue to negatively affect those populations; you are doing the loggers and dam builders a favor by blaming fishing, which is far easier (and more politically palatable) to regulate.
• phkahler 3y ago
  >> This sounds fishy. Sounds like their primary result was nothing If you combine results from a bunch of studies that really didn't do anything, it's going to mask the results from the ones that did.
  • mlyle 3y ago
    Sure... but conversely, if you keep dividing up and analyzing in different ways, eventually you'll get a p<0.05 result. This is why you "call your shot" before beginning the analysis. Otherwise, people will be suspicious you just jiggered things around until you found something.
  • oldgradstudent 3y ago
    Maybe, but with 14 studies you have ~2^14 ways to split the studies into two groups. If you test multiple hypotheses, you have too adjust the p-value accordingly. Have the authors disclosed how many alternative hypotheses were tested until the result in the article was found to be significant?
    • anonymouskimmer 3y ago
      P-values don't prove anything about whether an effect is real or not. They, at most, provide some degree of confidence with the presumption that the independent and dependent variables are indeed independent and dependent. P-values should, at most, be used to direct further mechanistic studies (if possible). Only use them on their lonesome if this hasn't yet been done, or isn't possible. And if that's the case, reverify them independently (such as by doing another meta-analysis using different data).
• deleted 3y ago
  [deleted]
• tuukkah 3y ago
  > This systematic review was registered in PROSPERO before data collection to preclude data-driven analyses and selective reporting (CRD42020185566). In addition, the methods, including the selection criteria, the statistical analysis, outcomes, and subgroup and sensitivity analyses, were published in advance in a study protocol (Schöttker et al., 2021). EDIT: And in the protocol, the first subgroup analysis they list is relevant: > Daily dose versus weekly/monthly bolus dose versus bolus dose at the beginning of the trial followed by a daily dose https://bmjopen.bmj.com/content/11/1/e041607 https://bmjopen.bmj.com/content/11/1/e041607
• cscheid 3y ago
  Yes, exactly. There's a real-life example of an author group who got sufficiently annoyed at a reviewer requesting an inappropriate subgroup analysis: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(00)02031-6/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... Reviewers asked for the subgroup analysis; authors said "no, this is statistically nonsense"; reviewers said "yes, but we'll reject the paper otherwise"; reviewers said "ok, but only if you let us also split on astrological sign". Result: paper reports that aspirin has an effect, but only if you're not a Gemini or Libra. Too good.
  • anonymouskimmer 3y ago
    The two signs bracketing the summer months (I presume this was in the northern hemisphere?). That's a potentially interesting finding. > inappropriate subgroup analysis No subgroup is inappropriate unless you know all of the values of all of the parameters. But sure, the intent they tried to demonstrate (take all sub-group analyses with a grain of salt) is a good thing to remember. Edit: To whoever downvoted me, time of year of birth does have real-world effects. https://www.livescience.com/13958-birth-month-health-effects.html https://www.livescience.com/13958-birth-month-health-effects... > Previous studies have found similar links between spring births and various disorders, including schizophrenia, multiple sclerosis and even Type 1 diabetes. It's possible these diseases are linked to some environmental influence during gestation or the first few months of life, though researchers aren't sure what that could be. > The leading candidates including vitamin D levels, infections that come and go seasonally, changes in nutrition, and even possibly weather fluctuations, Handunnetthi told LiveScience. Now perhaps all of this is just bad science and these correlations are just statistical anomalies. But perhaps they aren't.
    • cscheid 3y ago
      > No subgroup is inappropriate unless you know all of the values of all of the parameters I don't know how to put it less bluntly: you're incorrect. > But sure, the intent they tried to demonstrate (take all sub-group analyses with a grain of salt) That's not what they intended to demonstrate. They intended to demonstrate that you need a _reason_ to want to split, and that reason needs to be given _ahead_ of the analysis. If you see the results _and then choose_ a new data analysis (that includes new subgroup analyses), your procedure is no longer statistically sound. This is a specifically bad statistical practice called HARKing https://en.wikipedia.org/wiki/HARKing https://en.wikipedia.org/wiki/HARKing
• itsoktocry 3y ago
  >Would require some checking if they had a preregistration, if this was registered as a secondary resuld and if they had done a proper statistica analysis of multiple outocmes. You should do that. Meanwhile, I'm going to (continue to) take a daily vitamin D, because I recognize not everything can be confirmed by a double-blind experiment, and the cost is low.
  • deleted 3y ago
    [deleted]
  • pella 3y ago
    > "I'm going to (continue to) take a daily vitamin D, because I recognize not everything can be confirmed by a double-blind experiment, and the cost is low." Or it may be possible to prove the opposite, as with the parachute. bmj: "Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial" https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094 Results: Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P>0.9). This finding was consistent across multiple subgroups. Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0.6 m for participants v mean of 9146 m for non-participants; P<0.001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P<0.001). "Conclusions: Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention. However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps. When beliefs regarding the effectiveness of an intervention exist in the community, randomized trials might selectively enroll individuals with a lower perceived likelihood of benefit, thus diminishing the applicability of the results to clinical practice." disclaimer: I am pro-Parachute! I believe 100% in the effectiveness of the parachute!
• bitL 3y ago
  Simpson's paradox is well known - aggregate often has different statistical properties than individual parts.
• trevorkoob 3y ago
  15 minutes of full-body UV exposure from the sun is estimated to provide the equivalent of 20-30,000 IU Vitamin D3. There is a definite need for studies that better determine the effective dose for D3. My research led me to the conclusion that studies should be done based not on standardized supplementation, but instead supplementation to a standardized blood serum level.