7 ms·

If the difference was statistically significant, sure. But it isn't, is it? ~50% of the time, the vaccinated group will have more deaths by chance, even if the

by csee 5y ago

If the difference was statistically significant, sure. But it isn't, is it? ~50% of the time, the vaccinated group will have more deaths by chance, even if the vaccine is safe - assuming either little to no community spread during the study, or a small study of young people. All this result shows is that the vaccine isn't effective at preventing car crashes, which is something we already knew.

beagle3 5y ago
I urge you to actually read the report - https://www.fda.gov/media/151733/download https://www.fda.gov/media/151733/download on page 23 has the latest counts (21:17) and no exact list of causes, but an earlier version https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.supplementary-material https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v... which only lists 15:14 deaths, page 12 does list causes: There are 4 deaths from cardiac arrest in the vaccine arm, vs. 1 in the placebo; and 1 death from covid vs. 2 in the placebo. Taken altogether, especially with what we now know about myocarditis/pericarditis risk from the vaccine, non trivial input. And I don't know if you're aware, but the original (dec 2020) approval basically rested on 4:1 placebo:vaccine severe cases. It was approved based on statistically insignificant data, but somehow the same counts of death is not even interesting. Depending on your model, 21:17 may or may not be statistically significant. I didn't see any "official" modeling of that, especially given that "no deaths were attributed to the vaccine" - if your model says something can't happen, it obviously can't statistically happen ..... There is one "death" (which might as well be a car crash) out of the 29=(15+14) causes listed. All others are medical.
- deleted 5y ago
  [deleted]
- csee 5y ago
  It is not close to statistically significant. Here are two monte carlos under different assumptions which checks how often we expect 21 or more deaths by chance under the null hypothesis of no-effect on non-Covid mortality (i.e., that the vaccines are safe). ## >>> import numpy >>> import random >>> N_vaccine_deaths = [] >>> for j in range(500): ... x = [] ... x = [0 for i in range(46000)] + [1 for i in range(38)] # 38 deaths, ~46000 non-deaths ... random.shuffle(x) ... N_vaccine_deaths += [np.sum(x[0:int(len(x)/2)])] >>> N_vaccine_deaths = np.array(N_vaccine_deaths) >>> p = np.sum(N_vaccine_deaths >= 21)/len(N_vaccine_deaths ) >>> p 0.296 >>> x = [] >>> for i in range(5000): ... x += [sum([random.random() < (38/46000) for i in range(int(46000/2))])] >>> p = np.sum(np.array(x) >= 21)/len(x) >>> p 0.3446 Even just looking at cardiac deaths only and it turns out to be statistically insignificant with a p-value around 0.2. And this is actually p-hacking since we didn't prespecify and lock-in that hypothesis before the study began. > And I don't know if you're aware, but the original (dec 2020) approval basically rested on 4:1 placebo:vaccine severe cases. It was approved based on statistically insignificant data, but somehow the same counts of death is not even interesting. I don't see how this is relevant. That was a long time ago and we didn't have much data, hence low statistical power, and the need to make a decision with an incomplete picture. We now have much more data from over 20 countries, and are able to draw more conclusive and confident conclusions about efficacy.
  - beagle3 5y ago
    Simulation is much appreciated, even if I would use a different method (but I didn’t have the time, so I’ll accept yours) We do not have efficacy data from 20 countries. E.g. israeli data which I am intimately familiar with is incredibly biased to show vaccine efficiency. Official data touted based on observational data showed 95% efficiency, while at the same time airport data which is the only place with unbiased testing showed about 60%. (That data is biased by demographics, but not by sampling). As you said, there’s no statistically significant RCT data showing vaccines are dangerous, but also no RCT data showing they are effective in preventing severe disease. Statistical significance cuts both ways.
    - csee 5y ago
      > E.g. israeli data which I am intimately familiar with is incredibly biased Can you expand on this? Why is the data biased by demographics? Is this data biased in the same way as the other countries that show a 90% effectiveness? As I argued above, there is a bias that's working against vaccines, given that a larger proportion of old people are vaccinated which leads to an understatement of their effectiveness. > while at the same time airport data which is the only place with unbiased testing showed about 60% Link? > [there's] no RCT data showing they are effective in preventing severe disease Only because they haven't done an RCT that could possibly pick that up, probably because of some ethical argument that giving people a placebo may cause them to behave more recklessly (which I would disagree with).
      - beagle3 5y ago
        > Is this data biased in the same way as the other countries that show a 90% effectiveness? Well, Israeli MOH rules are basically that unvaccinated people must test at least twice a week to continue normal life, whereas vaccinated people need not test even while showing symptoms, even when being admitted to a hospital ; This had variation over time (e.g., since Jan 2021, approx 8 months only unvaccinated had to test on hospital admission, and 4 months everyone), but that's the gist. Furthermore, upon exposure to a verified case, the vaccinated can do a home test, and self report that they are negative, and not quarantine - whereas the unvaccinated must quarantine, and test in an offical supervised test, twice in a 7-day period in order to finish quarantine after 7 days, or spend 14 days in quarantine if they do not want to test. (This was shortened to 5 days last week, and also had some variations over time, but same gist). There are many other biases. It's kind of ridiculous. I don't know exactly how other countries run these things, but I suspect it's similar in other places, though the details differ. >> while at the same time airport data which is the only place with unbiased testing showed about 60% > Link? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 It's biased by economical demographic traits, because the people who actually fly are not a representative sample; But since everyone who lands gets tested, there is no bias such as the ones I described above. (Well ... there is; a vaccinated person tests once on arrival and that's it; an unvaccinated person must test again in 10 days, and if positive in the 10-days after arrival is considered to have been postiive on arrival -- but compared to the above, this is a "small" bias). > Only because they haven't done an RCT that could possibly pick that up, Well, yes, but there isn't one. As I said, it cuts both ways - they haven't done a test that's powered enough to show vaccine ACM is worse than placebo ACM. Is that the only reason it doesn't exist? Maybe yes, maybe not. Cuts both ways. Are you familiar with Fenton & Neil's work? https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-efficacy-statistical-illusion.html https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-ef... https://probabilityandlaw.blogspot.com/2021/12/the-impact-of-misclassifying-deaths-in.html https://probabilityandlaw.blogspot.com/2021/12/the-impact-of... They also have academic level papers, but the blog posts are short and concise.
s1artibartfast 5y ago
Like you said, it all depends on the statistics. If the car crashes in the treated group are high enough, you have to start asking if there is a mechanism where vaccination side effects are the cause. This is why you don't drive or operate heavily machinery with certain drugs.

by csee 5y ago

beagle3 5y ago

I urge you to actually read the report - https://www.fda.gov/media/151733/download https://www.fda.gov/media/151733/download on page 23 has the latest counts (21:17) and no exact list of causes, but an earlier version https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.supplementary-material https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v... which only lists 15:14 deaths, page 12 does list causes: There are 4 deaths from cardiac arrest in the vaccine arm, vs. 1 in the placebo; and 1 death from covid vs. 2 in the placebo. Taken altogether, especially with what we now know about myocarditis/pericarditis risk from the vaccine, non trivial input. And I don't know if you're aware, but the original (dec 2020) approval basically rested on 4:1 placebo:vaccine severe cases. It was approved based on statistically insignificant data, but somehow the same counts of death is not even interesting. Depending on your model, 21:17 may or may not be statistically significant. I didn't see any "official" modeling of that, especially given that "no deaths were attributed to the vaccine" - if your model says something can't happen, it obviously can't statistically happen ..... There is one "death" (which might as well be a car crash) out of the 29=(15+14) causes listed. All others are medical.

deleted 5y ago
[deleted]
csee 5y ago
It is not close to statistically significant. Here are two monte carlos under different assumptions which checks how often we expect 21 or more deaths by chance under the null hypothesis of no-effect on non-Covid mortality (i.e., that the vaccines are safe). ## >>> import numpy >>> import random >>> N_vaccine_deaths = [] >>> for j in range(500): ... x = [] ... x = [0 for i in range(46000)] + [1 for i in range(38)] # 38 deaths, ~46000 non-deaths ... random.shuffle(x) ... N_vaccine_deaths += [np.sum(x[0:int(len(x)/2)])] >>> N_vaccine_deaths = np.array(N_vaccine_deaths) >>> p = np.sum(N_vaccine_deaths >= 21)/len(N_vaccine_deaths ) >>> p 0.296 >>> x = [] >>> for i in range(5000): ... x += [sum([random.random() < (38/46000) for i in range(int(46000/2))])] >>> p = np.sum(np.array(x) >= 21)/len(x) >>> p 0.3446 Even just looking at cardiac deaths only and it turns out to be statistically insignificant with a p-value around 0.2. And this is actually p-hacking since we didn't prespecify and lock-in that hypothesis before the study began. > And I don't know if you're aware, but the original (dec 2020) approval basically rested on 4:1 placebo:vaccine severe cases. It was approved based on statistically insignificant data, but somehow the same counts of death is not even interesting. I don't see how this is relevant. That was a long time ago and we didn't have much data, hence low statistical power, and the need to make a decision with an incomplete picture. We now have much more data from over 20 countries, and are able to draw more conclusive and confident conclusions about efficacy.
- beagle3 5y ago
  Simulation is much appreciated, even if I would use a different method (but I didn’t have the time, so I’ll accept yours) We do not have efficacy data from 20 countries. E.g. israeli data which I am intimately familiar with is incredibly biased to show vaccine efficiency. Official data touted based on observational data showed 95% efficiency, while at the same time airport data which is the only place with unbiased testing showed about 60%. (That data is biased by demographics, but not by sampling). As you said, there’s no statistically significant RCT data showing vaccines are dangerous, but also no RCT data showing they are effective in preventing severe disease. Statistical significance cuts both ways.
  - csee 5y ago
    > E.g. israeli data which I am intimately familiar with is incredibly biased Can you expand on this? Why is the data biased by demographics? Is this data biased in the same way as the other countries that show a 90% effectiveness? As I argued above, there is a bias that's working against vaccines, given that a larger proportion of old people are vaccinated which leads to an understatement of their effectiveness. > while at the same time airport data which is the only place with unbiased testing showed about 60% Link? > [there's] no RCT data showing they are effective in preventing severe disease Only because they haven't done an RCT that could possibly pick that up, probably because of some ethical argument that giving people a placebo may cause them to behave more recklessly (which I would disagree with).
    - beagle3 5y ago
      > Is this data biased in the same way as the other countries that show a 90% effectiveness? Well, Israeli MOH rules are basically that unvaccinated people must test at least twice a week to continue normal life, whereas vaccinated people need not test even while showing symptoms, even when being admitted to a hospital ; This had variation over time (e.g., since Jan 2021, approx 8 months only unvaccinated had to test on hospital admission, and 4 months everyone), but that's the gist. Furthermore, upon exposure to a verified case, the vaccinated can do a home test, and self report that they are negative, and not quarantine - whereas the unvaccinated must quarantine, and test in an offical supervised test, twice in a 7-day period in order to finish quarantine after 7 days, or spend 14 days in quarantine if they do not want to test. (This was shortened to 5 days last week, and also had some variations over time, but same gist). There are many other biases. It's kind of ridiculous. I don't know exactly how other countries run these things, but I suspect it's similar in other places, though the details differ. >> while at the same time airport data which is the only place with unbiased testing showed about 60% > Link? https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3963606 It's biased by economical demographic traits, because the people who actually fly are not a representative sample; But since everyone who lands gets tested, there is no bias such as the ones I described above. (Well ... there is; a vaccinated person tests once on arrival and that's it; an unvaccinated person must test again in 10 days, and if positive in the 10-days after arrival is considered to have been postiive on arrival -- but compared to the above, this is a "small" bias). > Only because they haven't done an RCT that could possibly pick that up, Well, yes, but there isn't one. As I said, it cuts both ways - they haven't done a test that's powered enough to show vaccine ACM is worse than placebo ACM. Is that the only reason it doesn't exist? Maybe yes, maybe not. Cuts both ways. Are you familiar with Fenton & Neil's work? https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-efficacy-statistical-illusion.html https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-ef... https://probabilityandlaw.blogspot.com/2021/12/the-impact-of-misclassifying-deaths-in.html https://probabilityandlaw.blogspot.com/2021/12/the-impact-of... They also have academic level papers, but the blog posts are short and concise.

s1artibartfast 5y ago

Like you said, it all depends on the statistics. If the car crashes in the treated group are high enough, you have to start asking if there is a mechanism where vaccination side effects are the cause. This is why you don't drive or operate heavily machinery with certain drugs.