8 ms·

> The rate of major cardiovascular events was lower in the vitamin D group than in the placebo group (hazard ratio 0.91, 95% confidence interval 0.81 to 1.01)

> The rate of major cardiovascular events was lower in the vitamin D group than in the placebo group (hazard ratio 0.91, 95% confidence interval 0.81 to 1.01) So, null result?

FollowingTheDao 3y ago
It’s not a null result for the people for which vitamin D supplementation prevented their heart attack. Just because it had an insignificant result for a genetically random population does not mean it did not help any individual in the study.
- artinmg 3y ago
  The issue is that with this result we cannot say that anyone was helped. The results are consistent with a null finding—that any effect measured is likely due to chance alone.
  - caf 3y ago
    The results are consistent with a null finding This is true. -that any effect measured is likely due to chance alone. But this is a stronger statement that is not. The 95% confidence interval for the hazard ratio was 0.81 to 1.01 which means there is more than a 5% probability that the result was due to chance, but not that it's likely due to chance, which implies "more probable than not". The result is more like: if the measured effect is real, it's small enough that a significantly larger study would be needed to demonstrate it with sufficient confidence.
    - artinmg 3y ago
      Fair enough, I phrased it inaccurately, although “likely” is subjective. I should say it’s reasonable to conclude that the results could be due to chance alone. At any rate the results do not tell us that anyone, in fact, benefited from vitamin D supplementation with respect to CVD risk.
- adrianmonk 3y ago
  > It’s not a null result for the people for which vitamin D supplementation prevented their heart attack. That's a true statement, but it's also not known whether there are any such people.
marcosdumay 3y ago
Looks like so. And it's bigger than that. It's one of the few large and well controlled studies done on the subject. It invalidates most of what was known about it (even though AFAIK, it doesn't contradict them).
lumb63 3y ago
I don’t think this is the right way of looking at the result. Imagine you have cardiovascular disease. If you take vitamin D, the study suggests that you will on average a 9% reduction in risk. There is a ~97% chance that it provides you some improvement, but there is a ~3% chance that your risk actually increases. This seems like an easy decision to me if you are a high risk individual. Looked at another way, imagine the authors used a 94% confidence interval instead of 95%. Suddenly, the risk reduction from vitamin D becomes “statistically significant”. There is nothing magical about 95%, and use of p values to assess boolean effective/ineffective should have been done away with a long time ago. Related: The Earth is Round (p < 0.05) [1994]: https://www.sjsu.edu/faculty/gerstman/misc/Cohen1994.pdf https://www.sjsu.edu/faculty/gerstman/misc/Cohen1994.pdf Sadly, the Earth is Still Round (p < 0.05) [2012]: https://cmapspublic2.ihmc.us/rid=1P501PBWQ-1S6L809-28MX/Zhu_2012_Sadly,%20the%20earth%20is%20still%20round%20(p%200.pdf https://cmapspublic2.ihmc.us/rid=1P501PBWQ-1S6L809-28MX/Zhu_...

> The rate of major cardiovascular events was lower in the vitamin D group than in the placebo group (hazard ratio 0.91, 95% confidence interval 0.81 to 1.01) So, null result?

FollowingTheDao 3y ago

It’s not a null result for the people for which vitamin D supplementation prevented their heart attack. Just because it had an insignificant result for a genetically random population does not mean it did not help any individual in the study.

artinmg 3y ago
The issue is that with this result we cannot say that anyone was helped. The results are consistent with a null finding—that any effect measured is likely due to chance alone.
- caf 3y ago
  The results are consistent with a null finding This is true. -that any effect measured is likely due to chance alone. But this is a stronger statement that is not. The 95% confidence interval for the hazard ratio was 0.81 to 1.01 which means there is more than a 5% probability that the result was due to chance, but not that it's likely due to chance, which implies "more probable than not". The result is more like: if the measured effect is real, it's small enough that a significantly larger study would be needed to demonstrate it with sufficient confidence.
  - artinmg 3y ago
    Fair enough, I phrased it inaccurately, although “likely” is subjective. I should say it’s reasonable to conclude that the results could be due to chance alone. At any rate the results do not tell us that anyone, in fact, benefited from vitamin D supplementation with respect to CVD risk.
adrianmonk 3y ago
> It’s not a null result for the people for which vitamin D supplementation prevented their heart attack. That's a true statement, but it's also not known whether there are any such people.

marcosdumay 3y ago

Looks like so. And it's bigger than that. It's one of the few large and well controlled studies done on the subject. It invalidates most of what was known about it (even though AFAIK, it doesn't contradict them).

lumb63 3y ago

I don’t think this is the right way of looking at the result. Imagine you have cardiovascular disease. If you take vitamin D, the study suggests that you will on average a 9% reduction in risk. There is a ~97% chance that it provides you some improvement, but there is a ~3% chance that your risk actually increases. This seems like an easy decision to me if you are a high risk individual. Looked at another way, imagine the authors used a 94% confidence interval instead of 95%. Suddenly, the risk reduction from vitamin D becomes “statistically significant”. There is nothing magical about 95%, and use of p values to assess boolean effective/ineffective should have been done away with a long time ago. Related: The Earth is Round (p < 0.05) [1994]: https://www.sjsu.edu/faculty/gerstman/misc/Cohen1994.pdf https://www.sjsu.edu/faculty/gerstman/misc/Cohen1994.pdf Sadly, the Earth is Still Round (p < 0.05) [2012]: https://cmapspublic2.ihmc.us/rid=1P501PBWQ-1S6L809-28MX/Zhu_2012_Sadly,%20the%20earth%20is%20still%20round%20(p%200.pdf https://cmapspublic2.ihmc.us/rid=1P501PBWQ-1S6L809-28MX/Zhu_...