3 ms·

N = 1500 One sample had 79-86 out of 750 hospitalized, and the other had 117-123 out of 750. Does that seem sufficient to demonstrate statistical significance

N = 1500 One sample had 79-86 out of 750 hospitalized, and the other had 117-123 out of 750. Does that seem sufficient to demonstrate statistical significance? Edit: found the actual numbers in the study 79 [11%] of 741 vs 119 [16%] of 756

SkittyDog 5y ago
Yes, as a matter of fact, it does demonstrate statistical significance.
- darkerside 5y ago
  For what minimum detectable effect?
  - goldemerald 5y ago
    You'd be interested in learning about the origin and application of Chi-Squared tests (or Fisher's exact test). Detecting changes in population means, and making sure those changes aren't just random noise, is quite straightforward and can be done by hand.
  - SkittyDog 5y ago
    It won't be possible for me to explain the answer to your satisfaction, if you lack at least a basic education in probability & statistics. Maybe Khan Academy has an intro level course, or a "Stats for Humanities" variant that can help you get started.
ekianjo 5y ago
Most drugs get approved with less strong effect than this
- darkerside 5y ago
  Well, of course they do. There's a terrific market for pharmaceuticals.
_3u10 5y ago
I ran it through an AB test significance calculator using the upper bound for each. p value was 0.9972 and the power was 92.40% it's a significant result. "Variant B’s conversion rate (11.47%) was 30.08% lower than variant A’s conversion rate (16.40%). You can be 99% confident that variant B will perform worse than variant A."
- darkerside 5y ago
  > When the DSMC met on Aug 5, 2021, it recommended that the TOGETHER trial stop randomly assigning patients to the fluvoxamine group, as this comparison had met the prespecified superiority criterion for the primary endpoint (prespecified superiority threshold 97·6%). That seems problematic. Did they stop the study because they had achieved statistical significance?
  - scarmig 5y ago
    Another issue is that they were testing multiple drugs, none of which had a statistically significant effect. Classic p hacking. https://xkcd.com/882/ https://xkcd.com/882/
    - scotty79 5y ago
      Assesing 7 promising substances to determine which one if any does anything and getting 30% effect on one and no effect on the rest is p-hacking? Is science allowed to research only one substance for specific purpose at any time to sufficiently avoid p-hacking for you? Also if I read correctly they got >0.60 not >0.05
- Jabbles 5y ago
  You can be 99% confident To be more precise without being too formal: If there truly were no difference between them, you could be 99% confident that you would not get a result like this.
novaRom 5y ago
I wish every news media to show original numbers always, not percentage.

SkittyDog 5y ago

Yes, as a matter of fact, it does demonstrate statistical significance.

darkerside 5y ago
For what minimum detectable effect?
- goldemerald 5y ago
  You'd be interested in learning about the origin and application of Chi-Squared tests (or Fisher's exact test). Detecting changes in population means, and making sure those changes aren't just random noise, is quite straightforward and can be done by hand.
- SkittyDog 5y ago
  It won't be possible for me to explain the answer to your satisfaction, if you lack at least a basic education in probability & statistics. Maybe Khan Academy has an intro level course, or a "Stats for Humanities" variant that can help you get started.

ekianjo 5y ago

Most drugs get approved with less strong effect than this

darkerside 5y ago
Well, of course they do. There's a terrific market for pharmaceuticals.

_3u10 5y ago

I ran it through an AB test significance calculator using the upper bound for each. p value was 0.9972 and the power was 92.40% it's a significant result. "Variant B’s conversion rate (11.47%) was 30.08% lower than variant A’s conversion rate (16.40%). You can be 99% confident that variant B will perform worse than variant A."

darkerside 5y ago
> When the DSMC met on Aug 5, 2021, it recommended that the TOGETHER trial stop randomly assigning patients to the fluvoxamine group, as this comparison had met the prespecified superiority criterion for the primary endpoint (prespecified superiority threshold 97·6%). That seems problematic. Did they stop the study because they had achieved statistical significance?
- scarmig 5y ago
  Another issue is that they were testing multiple drugs, none of which had a statistically significant effect. Classic p hacking. https://xkcd.com/882/ https://xkcd.com/882/
  - scotty79 5y ago
    Assesing 7 promising substances to determine which one if any does anything and getting 30% effect on one and no effect on the rest is p-hacking? Is science allowed to research only one substance for specific purpose at any time to sufficiently avoid p-hacking for you? Also if I read correctly they got >0.60 not >0.05
Jabbles 5y ago
You can be 99% confident To be more precise without being too formal: If there truly were no difference between them, you could be 99% confident that you would not get a result like this.

novaRom 5y ago

I wish every news media to show original numbers always, not percentage.