29 ms·

The author explains the following, regarding Vitamin D studies: > Most studies follow this pattern: Two sets of people are evaluated. One set has a certain dis

by amgreg 6y ago

The author explains the following, regarding Vitamin D studies: > Most studies follow this pattern: Two sets of people are evaluated. One set has a certain disease (diabetes, for example). The other set does not have the disease. Vitamin D levels are measured in both groups. Vitamin D deficiency is found to be much more common in the group of diseased individuals. If this is true, I wonder if most studies aren’t falling short for failing to control for Vitamin D deficiency. Couldn’t the studies be structured differently? Let’s say we want to know the effects of Vitamin D on Covid. What if, instead of measuring Vitamin D deficiency in a group with the malady and a group without (analogous to what the author suggests most studies do): We had two groups made up of Vitamin D deficient people. We gave the first group a Vitamin D supplement and the other a placebo. We then observed both groups out in the wild (ideally a place with a high R), measuring for infections. If the supplement folks were infected at significantly lower rates than placebo folks, wouldn’t this be better at demonstrating causation?

jonny_eh 6y ago
> failing to control for Vitamin D deficiency Do you mean failing to control for the disease causing vitamin D deficiency? Because it seems that's the issue you're addressing with your proposed study design.
- pdr2020 6y ago
  I think the issue he's addressing with his design is whether supplementing vitamin D would actually CAUSE infection rates to fall - versus simply knowing that vitamin D levels and COVID infections are correlated.
- deleted 6y ago
  [deleted]
choxi 6y ago
To your point, a confounding variable could be that diseased people go outside less (and therefore have less vitamin D)
- ajsnigrutin 6y ago
  this! In slovenia, most of the deaths are in old-age/nursing homes, where most of the patients have severe health issues and a lot of them are bed-bound. Yes, they have lower vitamin D, but also many other health problem, so looking at a single variable (vitamin D) proves corellation but can't prove any causation without interfering (eg. giving half of them vitamin D supplements, to compare to the other half).
  - devaboone 6y ago
    Great point. Low Vitamin D is common in very sick people, partly because they are not as active, not going outside, and may not be eating the best diets. There are trials like you describe, which I'm hoping to get to in part 2... or 3? There is a ton of information to cover!
pdr2020 6y ago
Yes, but it would also be much harder to organise as a study and require significantly more resources. Unfortunately, most studies prefer less work/almost equally publishable/newsworthy results :)
deleted 6y ago
[deleted]
mft_ 6y ago
You’re describing observational vs. interventional studies. Such Interventional studies can be done, but would be more difficult and expensive, and might be difficult to get an answer from.
wjnc 6y ago
A quasi-fix could be to control for the most obvious sources of vitamin D: how often do you go outside, how long do you go outside, do you take vitamin D supplements, how often and with which strength. You can do this kind of analysis alongside any type of study, don't need full coverage and get some preliminary results. All you need is a Bayesian on your team (: partially in jest) or the willingness to open up your data after you publish. In my opinion as someone who practices non-medical statistics the trope of correlation vs. causation and interventional vs. observational are somewhat overblown. Andrew Gelman posted somewhat snarky about MDs and statistics recently [1]. I see that this post is in (very) good faith, but it does have the MD-bias to statistics in it. [1] https://statmodeling.stat.columbia.edu/2020/08/11/that-not-a-real-doctor-thing-its-kind-of-silly-for-people-to-think-that-going-to-medical-school-for-a-few-years-will-give-you-the-skills-necessary-to-be-able-to-evaluate-research-claims-in/ https://statmodeling.stat.columbia.edu/2020/08/11/that-not-a...
- devaboone 6y ago
  I'm the one who wrote the blog, and really appreciate your comment. As a physician, I can confirm that most physicians have only the tiniest grasp on statistics - and most will not have even heard about Bayesian stats. I do not want to imply that observational studies of correlation are useless - or even that observational studies are "less valid" than randomized controlled trials (something that doctors usually assume, but it incorrect). I actually want to write about this issue in future posts. There is a lot I want to cover, and stats in medicine is one of my favorite topics. One blog at a time, though.
  - wjnc 6y ago
    I'm looking forward to future posts as well! Thanks for writing this. You gave me pause to reconsider the only supplement I give my children and perhaps even more so in future episodes.
jwally 6y ago
The quote you cite is interesting if not a little worrisome. Maybe its just acting as a first-filter of things to (not) investigate at more expense later? Otherwise you'd get a ton of studies promoting solutions as cause-and-effect when they're only coincidentally related: - U.S. Spending on science correlates @99.8% with suicide by hanging - Drownings correlates with Nic Cage films @66% - Japanese passenger cars sold in US correlates with suicide by motor-vehicle @ 93.57% *source: https://tylervigen.com/spurious-correlations https://tylervigen.com/spurious-correlations
- devaboone 6y ago
  I love this Spurious Correlations site. And you are correct - a correlation can be a first-filter, especially if you have some reason to believe that there would be a causative connection. And though "correlation does not equal causation" is a cliche, it is still so easy to get sucked into equating the two. Part of the human brain's desire to make sense of things.
  - kwillets 6y ago
    Correlation does not equal causation, but it correlates with it.
VLM 6y ago
As a very specific example I think you're looking for this vit D study. https://pubmed.ncbi.nlm.nih.gov/23032549/ https://pubmed.ncbi.nlm.nih.gov/23032549/ As a very general example I think you'd have a lot of fun looking at examine.com pages that link to medical studies. https://examine.com/supplements/vitamin-d/ https://examine.com/supplements/vitamin-d/ Two axis of engineering problem definition not handled by the article are definition of normal and lifestyle variation. The concept of normal blood chemistry levels is vary vague. Certainly for mariners over 200 years ago the level of vitamin C was always normally very low. Running a nutrition program for Columbus-era mariners to optimize their diet to produce the most numerically average possible vit-C level would be possible but would not be healthy at all. There's a giant subculture of both doctors and average people running all kinds of semi-long term experiments on diet and health. Another side dish is the definition of normal for diet, some consider paleo to be normal diet and some consider twinkies and hot pockets to be normal diet. And of course that "normal" diet interacts in peculiar combination with "normal" concept of blood chemistry making it a very complicated problem. The lifestyle issue is an interesting problem. Due to local weather I don't go outside and due to exercise hobbies I'm extremely large from weight lifting for many years, also I sweat out a ton of electrolytes and presumably water soluble vitamins every other day. A stereotypical elderly petite sedentary zero-exercise desk worker would likely turn into a pillar of salt if they consumed the same salts I require to prevent muscle cramps, but perhaps this theoretical person is old enough to still falsely consider sun tanning as a healthy activity. Meanwhile our evolutionary ancestors evolved to work as physically hard as me yet do it all out in the sunlight naked. Given this incredible diversity in lifestyle, you'd think we'd all take different vitamin/supplements much like we drink varying amounts of water, perhaps a 5 to 1 ratio of individual variation if not more, but instead the bottle of vit D pills in front of me claims we should all take 125 mcg aka 5000 IU which is 625% of the normal RDA. That seems highly unrealistic. I'm sure there's potential startup ideas to both gather data and analyze the data. Rather than running a trial on 322 people and hoping its good enough, you could gather less accurate data from 1e6 people, maybe 1e7 people, using some kind of cloudy app statistical sampling thingie.

by amgreg 6y ago

jonny_eh 6y ago

> failing to control for Vitamin D deficiency Do you mean failing to control for the disease causing vitamin D deficiency? Because it seems that's the issue you're addressing with your proposed study design.

pdr2020 6y ago
I think the issue he's addressing with his design is whether supplementing vitamin D would actually CAUSE infection rates to fall - versus simply knowing that vitamin D levels and COVID infections are correlated.
deleted 6y ago
[deleted]

choxi 6y ago

To your point, a confounding variable could be that diseased people go outside less (and therefore have less vitamin D)

ajsnigrutin 6y ago
this! In slovenia, most of the deaths are in old-age/nursing homes, where most of the patients have severe health issues and a lot of them are bed-bound. Yes, they have lower vitamin D, but also many other health problem, so looking at a single variable (vitamin D) proves corellation but can't prove any causation without interfering (eg. giving half of them vitamin D supplements, to compare to the other half).
- devaboone 6y ago
  Great point. Low Vitamin D is common in very sick people, partly because they are not as active, not going outside, and may not be eating the best diets. There are trials like you describe, which I'm hoping to get to in part 2... or 3? There is a ton of information to cover!

pdr2020 6y ago

Yes, but it would also be much harder to organise as a study and require significantly more resources. Unfortunately, most studies prefer less work/almost equally publishable/newsworthy results :)

deleted 6y ago

[deleted]

mft_ 6y ago

You’re describing observational vs. interventional studies. Such Interventional studies can be done, but would be more difficult and expensive, and might be difficult to get an answer from.

wjnc 6y ago

A quasi-fix could be to control for the most obvious sources of vitamin D: how often do you go outside, how long do you go outside, do you take vitamin D supplements, how often and with which strength. You can do this kind of analysis alongside any type of study, don't need full coverage and get some preliminary results. All you need is a Bayesian on your team (: partially in jest) or the willingness to open up your data after you publish. In my opinion as someone who practices non-medical statistics the trope of correlation vs. causation and interventional vs. observational are somewhat overblown. Andrew Gelman posted somewhat snarky about MDs and statistics recently [1]. I see that this post is in (very) good faith, but it does have the MD-bias to statistics in it. [1] https://statmodeling.stat.columbia.edu/2020/08/11/that-not-a-real-doctor-thing-its-kind-of-silly-for-people-to-think-that-going-to-medical-school-for-a-few-years-will-give-you-the-skills-necessary-to-be-able-to-evaluate-research-claims-in/ https://statmodeling.stat.columbia.edu/2020/08/11/that-not-a...

devaboone 6y ago
I'm the one who wrote the blog, and really appreciate your comment. As a physician, I can confirm that most physicians have only the tiniest grasp on statistics - and most will not have even heard about Bayesian stats. I do not want to imply that observational studies of correlation are useless - or even that observational studies are "less valid" than randomized controlled trials (something that doctors usually assume, but it incorrect). I actually want to write about this issue in future posts. There is a lot I want to cover, and stats in medicine is one of my favorite topics. One blog at a time, though.
- wjnc 6y ago
  I'm looking forward to future posts as well! Thanks for writing this. You gave me pause to reconsider the only supplement I give my children and perhaps even more so in future episodes.

jwally 6y ago

The quote you cite is interesting if not a little worrisome. Maybe its just acting as a first-filter of things to (not) investigate at more expense later? Otherwise you'd get a ton of studies promoting solutions as cause-and-effect when they're only coincidentally related: - U.S. Spending on science correlates @99.8% with suicide by hanging - Drownings correlates with Nic Cage films @66% - Japanese passenger cars sold in US correlates with suicide by motor-vehicle @ 93.57% *source: https://tylervigen.com/spurious-correlations https://tylervigen.com/spurious-correlations

devaboone 6y ago
I love this Spurious Correlations site. And you are correct - a correlation can be a first-filter, especially if you have some reason to believe that there would be a causative connection. And though "correlation does not equal causation" is a cliche, it is still so easy to get sucked into equating the two. Part of the human brain's desire to make sense of things.
- kwillets 6y ago
  Correlation does not equal causation, but it correlates with it.

VLM 6y ago

As a very specific example I think you're looking for this vit D study. https://pubmed.ncbi.nlm.nih.gov/23032549/ https://pubmed.ncbi.nlm.nih.gov/23032549/ As a very general example I think you'd have a lot of fun looking at examine.com pages that link to medical studies. https://examine.com/supplements/vitamin-d/ https://examine.com/supplements/vitamin-d/ Two axis of engineering problem definition not handled by the article are definition of normal and lifestyle variation. The concept of normal blood chemistry levels is vary vague. Certainly for mariners over 200 years ago the level of vitamin C was always normally very low. Running a nutrition program for Columbus-era mariners to optimize their diet to produce the most numerically average possible vit-C level would be possible but would not be healthy at all. There's a giant subculture of both doctors and average people running all kinds of semi-long term experiments on diet and health. Another side dish is the definition of normal for diet, some consider paleo to be normal diet and some consider twinkies and hot pockets to be normal diet. And of course that "normal" diet interacts in peculiar combination with "normal" concept of blood chemistry making it a very complicated problem. The lifestyle issue is an interesting problem. Due to local weather I don't go outside and due to exercise hobbies I'm extremely large from weight lifting for many years, also I sweat out a ton of electrolytes and presumably water soluble vitamins every other day. A stereotypical elderly petite sedentary zero-exercise desk worker would likely turn into a pillar of salt if they consumed the same salts I require to prevent muscle cramps, but perhaps this theoretical person is old enough to still falsely consider sun tanning as a healthy activity. Meanwhile our evolutionary ancestors evolved to work as physically hard as me yet do it all out in the sunlight naked. Given this incredible diversity in lifestyle, you'd think we'd all take different vitamin/supplements much like we drink varying amounts of water, perhaps a 5 to 1 ratio of individual variation if not more, but instead the bottle of vit D pills in front of me claims we should all take 125 mcg aka 5000 IU which is 625% of the normal RDA. That seems highly unrealistic. I'm sure there's potential startup ideas to both gather data and analyze the data. Rather than running a trial on 322 people and hoping its good enough, you could gather less accurate data from 1e6 people, maybe 1e7 people, using some kind of cloudy app statistical sampling thingie.