4 ms·

To be clear, this claim > For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half. is not one that can be justif

To be clear, this claim > For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half. is not one that can be justified by frequentist statistics such as those used in the article. We can't make probabilistic claims like that unless we go Bayesian. Well-designed drug trials typically plan in advance to have the sample size necessary to detect an effect of clinical significance. If, say, a 20% reduction in ARDS would be valuable, you would work out what sample size you'd need to reliably detect such a reduction. In this case, we simply don't have the sample size to conclude much of anything, because the confidence intervals include what I (as a non-physician, admittedly) see as a pretty wide range of clinically meaningful effect sizes, and we can't distinguish between them.

URSpider94 6y ago
Actually, I CAN make that claim based upon the data in the article. The relative risk [RR] of developing ARDS on HCQ vs. in the control group was 1.14, with 95% confidence interval of 0.65 - 2.00. That means that 95% of the time, the relative risk will fall between those two guardrails. If we take the single-sided probability, then the relative risk will be less 0.65 2.5% of the time. "Reducing the onset by half" means a relative risk of 0.5, which is less than the lower 95% confidence interval bound.
- capnrefsmmat 6y ago
  No, this is a standard example of a claim you cannot make with frequentist statistics, and can only make with a Bayesian credible interval. In frequentist statistics, the true population parameter is fixed, not random, so the claim "the relative risk will be less 0.65 2.5% of the time" has no meaning. The 95% claim is that if we repeatedly sample new data, 95% of the time we will generate a confidence interval covering the true parameter; but that does not say anything about the individual confidence interval in this specific case, about which we have no probability claim. Indeed, it's not clear what a 95% chance would mean for a specific interval, since probability in frequentist statistics is defined in terms of long-run probabilities over repeated samples of data. But the impulse to interpret confidence intervals in a Bayesian way is common, since it's the question we want confidence intervals to answer. You can find a lot of academic papers arguing about this as a sign of the inadequacy of frequentist statistics.

URSpider94 6y ago

Actually, I CAN make that claim based upon the data in the article. The relative risk [RR] of developing ARDS on HCQ vs. in the control group was 1.14, with 95% confidence interval of 0.65 - 2.00. That means that 95% of the time, the relative risk will fall between those two guardrails. If we take the single-sided probability, then the relative risk will be less 0.65 2.5% of the time. "Reducing the onset by half" means a relative risk of 0.5, which is less than the lower 95% confidence interval bound.

capnrefsmmat 6y ago
No, this is a standard example of a claim you cannot make with frequentist statistics, and can only make with a Bayesian credible interval. In frequentist statistics, the true population parameter is fixed, not random, so the claim "the relative risk will be less 0.65 2.5% of the time" has no meaning. The 95% claim is that if we repeatedly sample new data, 95% of the time we will generate a confidence interval covering the true parameter; but that does not say anything about the individual confidence interval in this specific case, about which we have no probability claim. Indeed, it's not clear what a 95% chance would mean for a specific interval, since probability in frequentist statistics is defined in terms of long-run probabilities over repeated samples of data. But the impulse to interpret confidence intervals in a Bayesian way is common, since it's the question we want confidence intervals to answer. You can find a lot of academic papers arguing about this as a sign of the inadequacy of frequentist statistics.