3 ms·
Whatever your opinion on metformin this study is not really providing strong enough evidence to take seriously. No one should give it any attention, except the
by ImaCake 5y ago
Whatever your opinion on metformin this study is not really providing strong enough evidence to take seriously. No one should give it any attention, except the researchers who will try and do a better one next time and hopefully can prove/disprove this important hypothesis.
The authors have done a lot of tests here. If you do enough tests, even on null data, you eventually get false positive results. Nevermind the effects of confounding which are difficult to control for effectively in a non-random study.
The smallest p-value I could find was P = 0.012. Which is for "elevated birth defect fre-
quencies among metformin-exposed offspring". This comes from a table with 6 rows of models with different OR values. Presumably they did have p-values testing if those ORs were statistically different from OR == 1. Which means one should adjust p-values by dividing the significance threshold by 6. So, 0.05/6 = 0.0083 which is smaller than 0.012. So I would be very hesitant to say that the evidence points to metformin really causing this. At least based on this study.
- rscho 5y agoThe bonferoni correction is known to be overly restrictive, though. But yes, probably many more models were fit and results are questionable. The author will at least be able to claim a Science paper... Also, adjustment of the p-value doesn't do anything to solve the flaws of hypothesis testing. But that's another debate entirely.
- arijun 5y agoRelevant xkcd: https://xkcd.com/882/ https://xkcd.com/882/