4 ms·

My layman's understanding was that "91%" "effective at preventing disease" (from the TFA) goes something like this: - Take two identical[1] populations of pe

My layman's understanding was that "91%" "effective at preventing disease" (from the TFA) goes something like this: - Take two identical[1] populations of people - Wait for some people (100 for easy math here) in the unvaccinated group to get the disease. Probably measured by a PCR test indicating presence of the virus and ignoring symptoms since they say "preventing disease" and not severe symptoms, hospitalization, death etc. - Count how many folks have the disease in the vaccinated group[2]. In the "91%" case we would expect 9 people to test positive, meaning 91 of 100 people we would expected to get the disease did not. I like to think of it as I am ~1/10th as likely to get the disease as I would be if i had not been vaccinated. Is this what they mean in this? [1] Doing your best to create identical groups by controlling for population differences such as gender, age, and other known risk factors, [2] We need also to control for time in the study etc. [Lots of editing for formatting etc]

matwood 6y ago
> I like to think of it as I am ~1/10th as likely to get the disease as I would be if i had not been vaccinated. Same, and it gets less likely if you are in a group of vaccinated people since the chances of being exposed to someone who is sick goes down. Herd immunity at that point.
herrvogel- 6y ago
This short video[1] explains it quite well imo. They also make the interesting point that efficacy rates are not comparable because the studies were done at different times/locations with different spread of the virus in the population. [1]https://youtu.be/K3odScka55A https://youtu.be/K3odScka55A
- amichal 6y ago
  Reading the more technically detailed comments I think really reinforces throwaway894345's point. I understood what "91%" means (much better than non-techies in my circle) but assumed a lot incorrectly about the controls and the actual "end point" they are measuring. It seems lots of folks are saying the different studies use different sets of symptoms etc and not routine testing at intervals to identify those who have the disease. I suspect most of the public doesn't even know what 91% means. Combine that with being bad at evaluating risk in general. It mostly represents a 10x improvement in cases per capita per time, but I think telling folks that their risk went from 1 in 10,000 to 1 in 100,000 (or whatever) probably doesn't mean a lot. Folks just want "safe" or "not-safe" :(
thamer 6y ago
> I like to think of it as I am ~1/10th as likely to get the disease as I would be if i had not been vaccinated. > Is this what they mean in this? Yes, that's what they mean. And even for the vaccines that are ~60-70% effective, all of them have so far been 100% effective in preventing hospitalization and death. It's not binary, the vaccines don't make you completely immune to this virus but greatly reduce the impact it has. I thought this video explained it well: https://www.youtube.com/watch?v=K3odScka55A https://www.youtube.com/watch?v=K3odScka55A
- makomk 6y ago
  As I understand it, it turns out that none of the vaccines are 100% effective in preventing hospitalization and death, it's just really hard to measure this accurately in the kind of studies used for vaccine approval because there aren't enough people in them, especially people at high risk. The actual figure from the various large-scale rollouts is closer to 80% I think?
  - alistairSH 6y ago
    Yeah, the language I've seen is usually something like "...there were 0 deaths in the test group..." Which most people will read as "100% effective in preventing deaths" but technically there's no specific claim at preventing deaths - the "0 deaths" is effectively an anecdote.
    - Spivak 6y ago
      I would assume that even a claim about being 100% effective has some asterisks like “has a strong enough immune system” where “strong enough” means “unless you’ve already been diagnosed with having a weak immune system you’re fine.”
  - thamer 6y ago
    It's quite possible that some people will eventually contract COVID-19 and die from it, even several weeks after getting their last immunization shot (supposedly at a point when they're fully protected). It just didn't happen in any of the trials. If anyone wants to look at the raw phase 3 numbers: Pfizer numbers: 43,448 participants received injections (21,720 the vaccine and 21,728 the placebo); 8 cases of COVID-19 in the vaccinated group and 162 in the placebo group, 9 were severe (8 of those in the placebo group). Pfizer study: https://www.nejm.org/doi/full/10.1056/NEJMoa2034577?query=featured_home https://www.nejm.org/doi/full/10.1056/NEJMoa2034577?query=fe... Moderna numbers: 30,420 participants, evenly split with 15,210 in each group, over 96% got both injections. There were 185 symptomatic cases in the placebo group and 11 in the vaccinated group. 30 participants had severe cases including one causing death: all were in the placebo group. Moderna study: https://www.nejm.org/doi/full/10.1056/nejmoa2035389 https://www.nejm.org/doi/full/10.1056/nejmoa2035389
makomk 6y ago
Most (possibly all?) of the headline figures for how effective Covid vaccines are at preventing disease are based on the number of symptomatic cases, including mild symptoms. So it's not purely measured based on number of positive tests regardless of symptoms, but it's not just severe symptoms either.
sokoloff 6y ago
Two weeks past second dose, I’m taking a lot fewer precautions, as I expect is the case for the vaccinated trial groups (based on an N of 1 where I know they started taking fewer precautions while in the J&J trial). This is, of course, the entire point of getting vaccinated. That might make the vaccine even more effective than the study shows if the stats don’t back out risk adjustment (which they probably cannot do).
- jfengel 6y ago
  The CDC recommendations do substantially reduce the precautions you have to take once you're fully vaccinated. But note that wearing a mask is still required. At a technical level, you may still be able to spread the disease, even if you're fully vaccinated. And the vaccines aren't 100% effective, so even the minimal self protection you get from a mask is better than nothing. More importantly, I believe, is sociological: once there are a lot of people in public without masks, even if it were safe, a lot of people would take it as an excuse not to wear one. That leaves both them and others open to getting sick. The mask is a minimal inconvenience, and should be continued. If you want want to skip the mask among people you know to be vaccinated, in private, that's about as safe as anything ever gets in life. Even small, private gatherings with a cluster of unvaccinated people (i.e. people who all live together) are reasonably safe without masks for those who are vaccinated.
  - sokoloff 6y ago
    I fully believe the CDC is optimizing for the sociological aspects rather than biological aspects with respect to mask-wearing. I'll do it for many more months, because the sociology aspect is valid.
    - yellowapple 6y ago
      Yep. Plus, it's about as easy as it gets, and I get to conceal myself from facial recognition cameras. Win-win-win.
    - HWR_14 6y ago

matwood 6y ago

> I like to think of it as I am ~1/10th as likely to get the disease as I would be if i had not been vaccinated. Same, and it gets less likely if you are in a group of vaccinated people since the chances of being exposed to someone who is sick goes down. Herd immunity at that point.

herrvogel- 6y ago

This short video[1] explains it quite well imo. They also make the interesting point that efficacy rates are not comparable because the studies were done at different times/locations with different spread of the virus in the population. [1]https://youtu.be/K3odScka55A https://youtu.be/K3odScka55A

amichal 6y ago
Reading the more technically detailed comments I think really reinforces throwaway894345's point. I understood what "91%" means (much better than non-techies in my circle) but assumed a lot incorrectly about the controls and the actual "end point" they are measuring. It seems lots of folks are saying the different studies use different sets of symptoms etc and not routine testing at intervals to identify those who have the disease. I suspect most of the public doesn't even know what 91% means. Combine that with being bad at evaluating risk in general. It mostly represents a 10x improvement in cases per capita per time, but I think telling folks that their risk went from 1 in 10,000 to 1 in 100,000 (or whatever) probably doesn't mean a lot. Folks just want "safe" or "not-safe" :(

thamer 6y ago

> I like to think of it as I am ~1/10th as likely to get the disease as I would be if i had not been vaccinated. > Is this what they mean in this? Yes, that's what they mean. And even for the vaccines that are ~60-70% effective, all of them have so far been 100% effective in preventing hospitalization and death. It's not binary, the vaccines don't make you completely immune to this virus but greatly reduce the impact it has. I thought this video explained it well: https://www.youtube.com/watch?v=K3odScka55A https://www.youtube.com/watch?v=K3odScka55A

makomk 6y ago
As I understand it, it turns out that none of the vaccines are 100% effective in preventing hospitalization and death, it's just really hard to measure this accurately in the kind of studies used for vaccine approval because there aren't enough people in them, especially people at high risk. The actual figure from the various large-scale rollouts is closer to 80% I think?
- alistairSH 6y ago
  Yeah, the language I've seen is usually something like "...there were 0 deaths in the test group..." Which most people will read as "100% effective in preventing deaths" but technically there's no specific claim at preventing deaths - the "0 deaths" is effectively an anecdote.
  - Spivak 6y ago
    I would assume that even a claim about being 100% effective has some asterisks like “has a strong enough immune system” where “strong enough” means “unless you’ve already been diagnosed with having a weak immune system you’re fine.”
- thamer 6y ago
  It's quite possible that some people will eventually contract COVID-19 and die from it, even several weeks after getting their last immunization shot (supposedly at a point when they're fully protected). It just didn't happen in any of the trials. If anyone wants to look at the raw phase 3 numbers: Pfizer numbers: 43,448 participants received injections (21,720 the vaccine and 21,728 the placebo); 8 cases of COVID-19 in the vaccinated group and 162 in the placebo group, 9 were severe (8 of those in the placebo group). Pfizer study: https://www.nejm.org/doi/full/10.1056/NEJMoa2034577?query=featured_home https://www.nejm.org/doi/full/10.1056/NEJMoa2034577?query=fe... Moderna numbers: 30,420 participants, evenly split with 15,210 in each group, over 96% got both injections. There were 185 symptomatic cases in the placebo group and 11 in the vaccinated group. 30 participants had severe cases including one causing death: all were in the placebo group. Moderna study: https://www.nejm.org/doi/full/10.1056/nejmoa2035389 https://www.nejm.org/doi/full/10.1056/nejmoa2035389

makomk 6y ago

Most (possibly all?) of the headline figures for how effective Covid vaccines are at preventing disease are based on the number of symptomatic cases, including mild symptoms. So it's not purely measured based on number of positive tests regardless of symptoms, but it's not just severe symptoms either.

sokoloff 6y ago

Two weeks past second dose, I’m taking a lot fewer precautions, as I expect is the case for the vaccinated trial groups (based on an N of 1 where I know they started taking fewer precautions while in the J&J trial). This is, of course, the entire point of getting vaccinated. That might make the vaccine even more effective than the study shows if the stats don’t back out risk adjustment (which they probably cannot do).

jfengel 6y ago
The CDC recommendations do substantially reduce the precautions you have to take once you're fully vaccinated. But note that wearing a mask is still required. At a technical level, you may still be able to spread the disease, even if you're fully vaccinated. And the vaccines aren't 100% effective, so even the minimal self protection you get from a mask is better than nothing. More importantly, I believe, is sociological: once there are a lot of people in public without masks, even if it were safe, a lot of people would take it as an excuse not to wear one. That leaves both them and others open to getting sick. The mask is a minimal inconvenience, and should be continued. If you want want to skip the mask among people you know to be vaccinated, in private, that's about as safe as anything ever gets in life. Even small, private gatherings with a cluster of unvaccinated people (i.e. people who all live together) are reasonably safe without masks for those who are vaccinated.
- sokoloff 6y ago
  I fully believe the CDC is optimizing for the sociological aspects rather than biological aspects with respect to mask-wearing. I'll do it for many more months, because the sociology aspect is valid.
  - yellowapple 6y ago
    Yep. Plus, it's about as easy as it gets, and I get to conceal myself from facial recognition cameras. Win-win-win.
  - HWR_14 6y ago