3 ms·

> Researchers have noticed it for 13 years! From the linked Ghaemi et al 2024 meta-analysis You’re cherry picking papers. Others have already shared other stud

by Aurornis 10mo ago

> Researchers have noticed it for 13 years! From the linked Ghaemi et al 2024 meta-analysis You’re cherry picking papers. Others have already shared other studies showing no significant effects of Vitamin D intervention. For any popular supplement you can find someone publishing papers with miraculous results, showing huge effect sizes and significant outcomes. This has been going on for decades. With Omega-3s the larger the trial size, the smaller the outcome. The largest trials have shown very little to no detectable effect. I think a lot of people are skeptical about pharmaceuticals because they see the profit motive, but they let their guard down when researchers and supplement pushers who have their own motives start pushing flawed studies and cherry picked results. > In other words, if the effect on antidepressants vs placebo is ~0.4, and the effect of a placebo vs no placebo (just time) is ~0.7, that means the majority of the effect of antidepressants & OTC pain meds is due to placebo. You keep getting closer to understanding why these effect size studies are so popular with alternative medicine and supplement sellers: They’re so easy to misinterpret or to take out of context. According your numbers, taking Tylenol would be worse than placebo alone! 0.4 vs 0.7 Does this make any sense to you? It should make you pause and think that maybe this is more complicated than picking singular numbers and comparing them. In this domain of cherry picking studies and comparing effect sizes, you’ve reached a conclusion where Vitamin D is far and away more effective than anything, placebo is better than OTC pain medicines, and OTC pain meds are worse than placebo. It’s time for a reality check that maybe this methodology isn’t actually representative of reality. You’re writing at length as if these studies you picked are definitive and your numeric comparisons tell the whole story, but I don’t think you’ve stopped to consider if this is even realistic.

ncasenmare 10mo ago
> You’re cherry picking papers. I just picked the most recent meta-analysis I could find, which also specifically estimates the dose-response curve. (Since averaging the effect at 400 IU and 4000 IU doesn't make sense.) > Others have already shared other studies showing no significant effects of Vitamin D intervention. Yes, and the Ghaemi et al 2024 meta-analysis addresses the methodological problems in those earlier meta-analyses. (For example, they average the effects at vastly varying doses from 400 IU and 4000 IU) > According your numbers, taking Tylenol would be worse than placebo alone! 0.4 vs 0.7 No, I understand this fine. Taking Tylenol would give you active medication + placebo + time, which is 0.4 + 0.7 + X > *1.1.* Taking open-label placebo is just placebo + time = *0.7* + X. (Edit: Also, these aren't "my" numbers. They're from a major peer-reviewed study published in Nature, the highest-impact journal. I don't like "hey look at the credentials here", but I bring it up to note I'm not anti-science, see below paragraph) === Stepping back, I suspect the broader concern you have is you (correctly!) see that supplement/nutrition research is sketchy & full of grifters. And at the current moment, it seems to play into the hands of anti-establishment anti-science types. I agree, and I'll try to edit the tone of the article to avoid that. That said, there still is some good science (among the crap), and I think the better evidence is accumulating (at least for Vitamin D) that it's on par with traditional antidepressants, possibly more. I agree that much larger trials are required.
- svara 10mo ago
  > They're from a major peer-reviewed study published in Nature, the highest-impact journal. No, the domain name is nature.com because it's in a Nature Publishing Group journal, Scientific Reports, which is their least prestigious journal. It's a common mistake, and they do that on purpose, of course, to leverage the Nature brand. It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously.
  - directevolve 10mo ago
    It’s less prestigious because it doesn’t judge papers on novelty, only on technical accuracy. For incremental research like this, it is an appropriate choice. The lower prestige has no bearing on the accuracy of their findings.
  - MRtecno98 10mo ago
    > It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously. It's still peer reviewed, and as the sibling comment said, more applicable to this type of research. Also you now went from raising understandable objections to refusing the argument because it comes from a specific journal, which doesn't sound very scientific to me
    - svara 10mo ago
      You're right it isn't fair to reject someone's scientific argument just because they seem unfamiliar with how professional science works. We shouldn't have believed the study more if it actually had been in Nature. I don't think that's what I was saying, though. The issue in this thread was about taking a step back and looking at the overall plausibility of the conclusions, taking together multiple studies. I agree with the GP that the argument doesn't really pass the smell test. That's still the main issue, and it is something that people who don't understand scientific publishing struggle understanding/doing, because they lack the intuition for how certain results came about.
kadushka 10mo ago
the larger the trial size, the smaller the outcome I find this a bit surprising. Could there be something else affecting the accuracy of larger trials? Perhaps they are not as careful, or cutting corners somewhere?
- hirvi74 10mo ago
  Just my hypothesis, but I wonder if larger sample sizes provide a more diverse population. A study with 1000 individuals is likely a poor representation of a species of 8.2 billion. I understand that studies try to their best to use a diverse population, but I often question how successful many studies are at this endeavor.
  - kadushka 10mo ago
    use a diverse population If that's the case, we should question whether different homogeneous population groups respond differently to the substance under test. After all, we don't want to know the "average temperature of patients in a hospital", do we?
    - hirvi74 10mo ago
      > If that's the case, we should question whether different homogeneous population groups respond differently to the substance under test. In terms of psychological treatments, I am honestly in support of this. Many mental illnesses can have a cultural component to them. > After all, we don't want to know the "average temperature of patients in a hospital", do we? No, I don't think we do. Am I understanding you correctly?
- lamename 10mo ago
  Maybe. Those could be the case. But ignoring all confounding factors, this phenomenon is possible with numerical experiments alone. One of the meanings of "the Law of Small Numbers". Basically, the possibility that the small study was underpowered, and just lucky...then the large studies with more power are closer to the truth. https://en.wikipedia.org/wiki/Faulty_generalization https://en.wikipedia.org/wiki/Faulty_generalization

ncasenmare 10mo ago

> You’re cherry picking papers. I just picked the most recent meta-analysis I could find, which also specifically estimates the dose-response curve. (Since averaging the effect at 400 IU and 4000 IU doesn't make sense.) > Others have already shared other studies showing no significant effects of Vitamin D intervention. Yes, and the Ghaemi et al 2024 meta-analysis addresses the methodological problems in those earlier meta-analyses. (For example, they average the effects at vastly varying doses from 400 IU and 4000 IU) > According your numbers, taking Tylenol would be worse than placebo alone! 0.4 vs 0.7 No, I understand this fine. Taking Tylenol would give you active medication + placebo + time, which is 0.4 + 0.7 + X > *1.1.* Taking open-label placebo is just placebo + time = *0.7* + X. (Edit: Also, these aren't "my" numbers. They're from a major peer-reviewed study published in Nature, the highest-impact journal. I don't like "hey look at the credentials here", but I bring it up to note I'm not anti-science, see below paragraph) === Stepping back, I suspect the broader concern you have is you (correctly!) see that supplement/nutrition research is sketchy & full of grifters. And at the current moment, it seems to play into the hands of anti-establishment anti-science types. I agree, and I'll try to edit the tone of the article to avoid that. That said, there still is some good science (among the crap), and I think the better evidence is accumulating (at least for Vitamin D) that it's on par with traditional antidepressants, possibly more. I agree that much larger trials are required.

svara 10mo ago
> They're from a major peer-reviewed study published in Nature, the highest-impact journal. No, the domain name is nature.com because it's in a Nature Publishing Group journal, Scientific Reports, which is their least prestigious journal. It's a common mistake, and they do that on purpose, of course, to leverage the Nature brand. It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously.
- directevolve 10mo ago
  It’s less prestigious because it doesn’t judge papers on novelty, only on technical accuracy. For incremental research like this, it is an appropriate choice. The lower prestige has no bearing on the accuracy of their findings.
- MRtecno98 10mo ago
  > It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously. It's still peer reviewed, and as the sibling comment said, more applicable to this type of research. Also you now went from raising understandable objections to refusing the argument because it comes from a specific journal, which doesn't sound very scientific to me
  - svara 10mo ago
    You're right it isn't fair to reject someone's scientific argument just because they seem unfamiliar with how professional science works. We shouldn't have believed the study more if it actually had been in Nature. I don't think that's what I was saying, though. The issue in this thread was about taking a step back and looking at the overall plausibility of the conclusions, taking together multiple studies. I agree with the GP that the argument doesn't really pass the smell test. That's still the main issue, and it is something that people who don't understand scientific publishing struggle understanding/doing, because they lack the intuition for how certain results came about.

kadushka 10mo ago

the larger the trial size, the smaller the outcome I find this a bit surprising. Could there be something else affecting the accuracy of larger trials? Perhaps they are not as careful, or cutting corners somewhere?

hirvi74 10mo ago
Just my hypothesis, but I wonder if larger sample sizes provide a more diverse population. A study with 1000 individuals is likely a poor representation of a species of 8.2 billion. I understand that studies try to their best to use a diverse population, but I often question how successful many studies are at this endeavor.
- kadushka 10mo ago
  use a diverse population If that's the case, we should question whether different homogeneous population groups respond differently to the substance under test. After all, we don't want to know the "average temperature of patients in a hospital", do we?
  - hirvi74 10mo ago
    > If that's the case, we should question whether different homogeneous population groups respond differently to the substance under test. In terms of psychological treatments, I am honestly in support of this. Many mental illnesses can have a cultural component to them. > After all, we don't want to know the "average temperature of patients in a hospital", do we? No, I don't think we do. Am I understanding you correctly?
lamename 10mo ago
Maybe. Those could be the case. But ignoring all confounding factors, this phenomenon is possible with numerical experiments alone. One of the meanings of "the Law of Small Numbers". Basically, the possibility that the small study was underpowered, and just lucky...then the large studies with more power are closer to the truth. https://en.wikipedia.org/wiki/Faulty_generalization https://en.wikipedia.org/wiki/Faulty_generalization