3 ms·
I ran it through an AB test significance calculator using the upper bound for each. p value was 0.9972 and the power was 92.40% it's a significant result. "Va
by _3u10 5y ago
I ran it through an AB test significance calculator using the upper bound for each.
p value was 0.9972 and the power was 92.40% it's a significant result.
"Variant B’s conversion rate (11.47%) was 30.08% lower than variant A’s conversion rate (16.40%). You can be 99% confident that variant B will perform worse than variant A."
- darkerside 5y ago> When the DSMC met on Aug 5, 2021, it recommended that the TOGETHER trial stop randomly assigning patients to the fluvoxamine group, as this comparison had met the prespecified superiority criterion for the primary endpoint (prespecified superiority threshold 97·6%). That seems problematic. Did they stop the study because they had achieved statistical significance?
- scarmig 5y agoAnother issue is that they were testing multiple drugs, none of which had a statistically significant effect. Classic p hacking. https://xkcd.com/882/ https://xkcd.com/882/
- scotty79 5y agoAssesing 7 promising substances to determine which one if any does anything and getting 30% effect on one and no effect on the rest is p-hacking? Is science allowed to research only one substance for specific purpose at any time to sufficiently avoid p-hacking for you? Also if I read correctly they got >0.60 not >0.05
- Jabbles 5y agoYou can be 99% confident To be more precise without being too formal: If there truly were no difference between them, you could be 99% confident that you would not get a result like this.