4 ms·

The main problem is that you need a lot more data to make confident statements about severe cases than just about symptomatic cases. That 100% number of the Pfi

by mfsch 5y ago

The main problem is that you need a lot more data to make confident statements about severe cases than just about symptomatic cases. That 100% number of the Pfizer trial had a huge uncertainty interval. So even though severe cases are what you care most about, you usually don’t want to wait for a study that gives you good data on those and measure (and compare) symptomatic cases instead. Zeynep Tufekci discussed this issue of study endpoints in one of her recent articles: https://www.theinsight.org/p/vaccine-efficacy-statistical-power https://www.theinsight.org/p/vaccine-efficacy-statistical-po...

dataflow 5y ago
But couldn't they just tell people about the uncertainty? (like "90-98% effective"?) Or underestimate the efficacy (in some objective way, like "at least 90% effective"). Or something else, as long as they're ultimately measuring what matters and what people actually care about. It's not like that 100% with their sample sizes would realistically ever suddenly turn into 50%, would it? It seems just inherently wrong to measure the wrong thing and convey it with certainty than to measure the right thing and convey it with the associated uncertainty, does it not?
- mfsch 5y ago
  They actually had a table on the efficacy for severe cases in the supplemental material of the original Pfizer study (table S5 in [1]), but all it shows is that the study doesn’t really give any information on this. The 95% confidence interval given for severe cases after the first vaccine is 20.1–99.7%. From what I understand, it’s really a trade-off between waiting for proper data about the efficacy you’re most interested in, or starting to vaccinate people as soon as you have efficacy data for an endpoint that’s still meaningful but easier to reach. Having much fewer symptomatic cases is already a pretty good result, and I think it was expected that the protection would also extend to severe cases. [1]: https://www.nejm.org/doi/suppl/10.1056/NEJMoa2034577/suppl_file/nejmoa2034577_appendix.pdf https://www.nejm.org/doi/suppl/10.1056/NEJMoa2034577/suppl_f...

by mfsch 5y ago

dataflow 5y ago

But couldn't they just tell people about the uncertainty? (like "90-98% effective"?) Or underestimate the efficacy (in some objective way, like "at least 90% effective"). Or something else, as long as they're ultimately measuring what matters and what people actually care about. It's not like that 100% with their sample sizes would realistically ever suddenly turn into 50%, would it? It seems just inherently wrong to measure the wrong thing and convey it with certainty than to measure the right thing and convey it with the associated uncertainty, does it not?

mfsch 5y ago
They actually had a table on the efficacy for severe cases in the supplemental material of the original Pfizer study (table S5 in [1]), but all it shows is that the study doesn’t really give any information on this. The 95% confidence interval given for severe cases after the first vaccine is 20.1–99.7%. From what I understand, it’s really a trade-off between waiting for proper data about the efficacy you’re most interested in, or starting to vaccinate people as soon as you have efficacy data for an endpoint that’s still meaningful but easier to reach. Having much fewer symptomatic cases is already a pretty good result, and I think it was expected that the protection would also extend to severe cases. [1]: https://www.nejm.org/doi/suppl/10.1056/NEJMoa2034577/suppl_file/nejmoa2034577_appendix.pdf https://www.nejm.org/doi/suppl/10.1056/NEJMoa2034577/suppl_f...