4 ms·

It is not close to statistically significant. Here are two monte carlos under different assumptions which checks how often we expect 21 or more deaths by chance

by csee 5y ago

It is not close to statistically significant. Here are two monte carlos under different assumptions which checks how often we expect 21 or more deaths by chance under the null hypothesis of no-effect on non-Covid mortality (i.e., that the vaccines are safe). ## >>> import numpy >>> import random >>> N_vaccine_deaths = [] >>> for j in range(500): ... x = [] ... x = [0 for i in range(46000)] + [1 for i in range(38)] # 38 deaths, ~46000 non-deaths ... random.shuffle(x) ... N_vaccine_deaths += [np.sum(x[0:int(len(x)/2)])] >>> N_vaccine_deaths = np.array(N_vaccine_deaths) >>> p = np.sum(N_vaccine_deaths >= 21)/len(N_vaccine_deaths ) >>> p 0.296 >>> x = [] >>> for i in range(5000): ... x += [sum([random.random() < (38/46000) for i in range(int(46000/2))])] >>> p = np.sum(np.array(x) >= 21)/len(x) >>> p 0.3446 Even just looking at cardiac deaths only and it turns out to be statistically insignificant with a p-value around 0.2. And this is actually p-hacking since we didn't prespecify and lock-in that hypothesis before the study began. > And I don't know if you're aware, but the original (dec 2020) approval basically rested on 4:1 placebo:vaccine severe cases. It was approved based on statistically insignificant data, but somehow the same counts of death is not even interesting. I don't see how this is relevant. That was a long time ago and we didn't have much data, hence low statistical power, and the need to make a decision with an incomplete picture. We now have much more data from over 20 countries, and are able to draw more conclusive and confident conclusions about efficacy.

beagle3 5y ago
Simulation is much appreciated, even if I would use a different method (but I didn’t have the time, so I’ll accept yours) We do not have efficacy data from 20 countries. E.g. israeli data which I am intimately familiar with is incredibly biased to show vaccine efficiency. Official data touted based on observational data showed 95% efficiency, while at the same time airport data which is the only place with unbiased testing showed about 60%. (That data is biased by demographics, but not by sampling). As you said, there’s no statistically significant RCT data showing vaccines are dangerous, but also no RCT data showing they are effective in preventing severe disease. Statistical significance cuts both ways.
- csee 5y ago
  > E.g. israeli data which I am intimately familiar with is incredibly biased Can you expand on this? Why is the data biased by demographics? Is this data biased in the same way as the other countries that show a 90% effectiveness? As I argued above, there is a bias that's working against vaccines, given that a larger proportion of old people are vaccinated which leads to an understatement of their effectiveness. > while at the same time airport data which is the only place with unbiased testing showed about 60% Link? > [there's] no RCT data showing they are effective in preventing severe disease Only because they haven't done an RCT that could possibly pick that up, probably because of some ethical argument that giving people a placebo may cause them to behave more recklessly (which I would disagree with).
  - beagle3 5y ago
    > Is this data biased in the same way as the other countries that show a 90% effectiveness? Well, Israeli MOH rules are basically that unvaccinated people must test at least twice a week to continue normal life, whereas vaccinated people need not test even while showing symptoms, even when being admitted to a hospital ; This had variation over time (e.g., since Jan 2021, approx 8 months only unvaccinated had to test on hospital admission, and 4 months everyone), but that's the gist. Furthermore, upon exposure to a verified case, the vaccinated can do a home test, and self report that they are negative, and not quarantine - whereas the unvaccinated must quarantine, and test in an offical supervised test, twice in a 7-day period in order to finish quarantine after 7 days, or spend 14 days in quarantine if they do not want to test. (This was shortened to 5 days last week, and also had some variations over time, but same gist). There are many other biases. It's kind of ridiculous. I don't know exactly how other countries run these things, but I suspect it's similar in other places, though the details differ. >> while at the same time airport data which is the only place with unbiased testing showed about 60% > Link? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 It's biased by economical demographic traits, because the people who actually fly are not a representative sample; But since everyone who lands gets tested, there is no bias such as the ones I described above. (Well ... there is; a vaccinated person tests once on arrival and that's it; an unvaccinated person must test again in 10 days, and if positive in the 10-days after arrival is considered to have been postiive on arrival -- but compared to the above, this is a "small" bias). > Only because they haven't done an RCT that could possibly pick that up, Well, yes, but there isn't one. As I said, it cuts both ways - they haven't done a test that's powered enough to show vaccine ACM is worse than placebo ACM. Is that the only reason it doesn't exist? Maybe yes, maybe not. Cuts both ways. Are you familiar with Fenton & Neil's work? https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-efficacy-statistical-illusion.html https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-ef... https://probabilityandlaw.blogspot.com/2021/12/the-impact-of-misclassifying-deaths-in.html https://probabilityandlaw.blogspot.com/2021/12/the-impact-of... They also have academic level papers, but the blog posts are short and concise.

by csee 5y ago

beagle3 5y ago

Simulation is much appreciated, even if I would use a different method (but I didn’t have the time, so I’ll accept yours) We do not have efficacy data from 20 countries. E.g. israeli data which I am intimately familiar with is incredibly biased to show vaccine efficiency. Official data touted based on observational data showed 95% efficiency, while at the same time airport data which is the only place with unbiased testing showed about 60%. (That data is biased by demographics, but not by sampling). As you said, there’s no statistically significant RCT data showing vaccines are dangerous, but also no RCT data showing they are effective in preventing severe disease. Statistical significance cuts both ways.

csee 5y ago
> E.g. israeli data which I am intimately familiar with is incredibly biased Can you expand on this? Why is the data biased by demographics? Is this data biased in the same way as the other countries that show a 90% effectiveness? As I argued above, there is a bias that's working against vaccines, given that a larger proportion of old people are vaccinated which leads to an understatement of their effectiveness. > while at the same time airport data which is the only place with unbiased testing showed about 60% Link? > [there's] no RCT data showing they are effective in preventing severe disease Only because they haven't done an RCT that could possibly pick that up, probably because of some ethical argument that giving people a placebo may cause them to behave more recklessly (which I would disagree with).
- beagle3 5y ago
  > Is this data biased in the same way as the other countries that show a 90% effectiveness? Well, Israeli MOH rules are basically that unvaccinated people must test at least twice a week to continue normal life, whereas vaccinated people need not test even while showing symptoms, even when being admitted to a hospital ; This had variation over time (e.g., since Jan 2021, approx 8 months only unvaccinated had to test on hospital admission, and 4 months everyone), but that's the gist. Furthermore, upon exposure to a verified case, the vaccinated can do a home test, and self report that they are negative, and not quarantine - whereas the unvaccinated must quarantine, and test in an offical supervised test, twice in a 7-day period in order to finish quarantine after 7 days, or spend 14 days in quarantine if they do not want to test. (This was shortened to 5 days last week, and also had some variations over time, but same gist). There are many other biases. It's kind of ridiculous. I don't know exactly how other countries run these things, but I suspect it's similar in other places, though the details differ. >> while at the same time airport data which is the only place with unbiased testing showed about 60% > Link? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 It's biased by economical demographic traits, because the people who actually fly are not a representative sample; But since everyone who lands gets tested, there is no bias such as the ones I described above. (Well ... there is; a vaccinated person tests once on arrival and that's it; an unvaccinated person must test again in 10 days, and if positive in the 10-days after arrival is considered to have been postiive on arrival -- but compared to the above, this is a "small" bias). > Only because they haven't done an RCT that could possibly pick that up, Well, yes, but there isn't one. As I said, it cuts both ways - they haven't done a test that's powered enough to show vaccine ACM is worse than placebo ACM. Is that the only reason it doesn't exist? Maybe yes, maybe not. Cuts both ways. Are you familiar with Fenton & Neil's work? https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-efficacy-statistical-illusion.html https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-ef... https://probabilityandlaw.blogspot.com/2021/12/the-impact-of-misclassifying-deaths-in.html https://probabilityandlaw.blogspot.com/2021/12/the-impact-of... They also have academic level papers, but the blog posts are short and concise.