9 ms·

41,135 Total enrolled ~20,567 in each arm 162/20,567 cases in the placebo arm (~0.79%) 8/20,567 cases in the vaccine arm (~0.04%) Absolute risk reduction of

41,135 Total enrolled ~20,567 in each arm 162/20,567 cases in the placebo arm (~0.79%) 8/20,567 cases in the vaccine arm (~0.04%) Absolute risk reduction of ~0.75% (over the time period of the study, which is difficult to tell). Number-needed-to-Treat (NNT): 134 vaccinations to prevent 1 symptomatic case over the study duration. Keep in mind, this is over some time period. Annualized, these numbers would almost certainly increase. We don’t know how many days/weeks/months it took to reach 162 / 8 cases respectively (something between ~5-6 months and ~5 days)

arijun 6y ago
Can you help interpret those numbers? Are they good?
- ComputerGuru 6y ago
  Really, it just means a) it works, b) the study should go on longer to get more significant results.
  - timerol 6y ago
    How much longer? There is significant benefit to rolling out massive amounts of vaccine immediately. I know that any scientist worth their salt will almost always claim that more study is warranted. But we're playing against a clock measured in human lives right now
    - henrikschroder 6y ago
      And if this vaccine doesn't work or has nasty side effects, or even ends up killing people, most people will be very reluctant to take it, or any of the next better vaccines, and that would cause even more deaths. It's a hard problem, and even though the clock is ticking, we also need to get it right the first time.
  - robrenaud 6y ago
    That relative risk reduction is very signficant, not just statistically signficant, it is very practically signficant too.
    - colinmhayes 6y ago
      Sure, but the sample of confirmed positives is so small that it might not include some important subgroups. If only 1 at risk person in the placebo tested positive, and 0 in the vaccinated group did we haven't learned much.
- chadash 6y ago
  The relevant piece for most people to know is that based on data from a short time period, people were 20x less likely to get symptomatic COVID if they get the vaccine. The exact number will probably change a bit as more time goes by and more data comes in, but 20x is the ballpark. Things we still don't know: - Does the vaccine also stop you from contracting the virus asymptomatically and spreading it? People enrolled in the trial were only told to get tested if they showed symptoms. - How long does the immunity from the vaccine last? We won't know this for some time.
  - neom 6y ago
    I presume how long immunity lasts is based on how well and quickly the virus can mutate itself? Do we have any research on other Coronaviruses that might hint at how long immunity will last?
    - tsimionescu 6y ago
      That's part of it, but it also depends on the 'memory' of your immune system.
  - joshyeager 6y ago
    A new study of 185 COVID patients found that 90% of the subjects were still seropositive for neutralizing antibodies at the 6 to 8 month time points. B-cells specific to the Spike and to the nucleocapsid coronavirus proteins actually increased over a five-month period. Memory T cells appeared to have half-lives of at least five or six months in these patients, and helper T cells were completely stable over the entire period studied. https://www.biorxiv.org/content/10.1101/2020.11.15.383323v1 https://www.biorxiv.org/content/10.1101/2020.11.15.383323v1
acqq 6y ago
The study enrolled more, but slowly increased the number of vaccinated since the start (which was initially, I'd expect, 0) -- the second dose was still not received by all enrolled as of November 13: "The Phase 3 clinical trial of BNT162b2 began on July 27 and has enrolled 43,661 participants to date, 41,135 of whom have received a second dose of the vaccine candidate as of November 13, 2020." So there is no single date one can reasonably use 41,135 or ~20,567 for the efficacy calculations. As the observations continue (expected up to 2 years) there could be much more useful data there. I also guess the people who haven't received both doses but got sick before are separately counted. The only useful numbers at the moment we have are 8 and 162 (and 1 and 9 for the severe cases). And that we can assume that from all thousands vaccinated these 170 in total got sick by chance, and that the thousands were carefully enough selected to represent bigger population, and using that we can expect that the efficacy won't too dramatically change as more data come in, the same way as the polls can use smaller samples to conclude something about bigger population, if the sampling is careful enough. But some changes are expected. It appears it's still too early to know if those who are vaccinated could protect those who aren't, or if the vaccine protects only those who received it (1) -- but anyway we know that still not all of those who will be vaccinated will be protected from having a severe case: there was up to now one severe case in the vaccinated group, versus 9 severe cases in the placebo group, 11%. --------- 1) https://www.medscape.com/viewarticle/941030 https://www.medscape.com/viewarticle/941030 "In an ideal world, a vaccine would prevent infection entirely and, it follows, also prevent disease and severe disease. But this may be hard to achieve for a respiratory virus vaccine. Animal challenge data suggest that vaccinated animals may still be infected even if they don't experience symptoms. A vaccine that is able to reduce the severity of disease, even if it cannot prevent infection entirely, would obviously still have enormous public health value. Therefore, this is what trials target as their primary aim."
- acqq 6y ago
  Maybe a better link: https://www.medscape.com/viewarticle/941030?src=soc_tw_201118_mscpedt_news_mdscp_vaccine&faf=1 https://www.medscape.com/viewarticle/941030?src=soc_tw_20111...
tgb 6y ago
Is NNT a good number to use when evaluating effectiveness of vaccines during a pandemic? It'll change substantially over time as the virus waxes and wanes and from location to location. So it won't be comparable across different vaccines. Moreover, the fact that cases create more cases means that preventing 1 case could prevent other future ones.
- virtuallynathan 6y ago
  I would agree, it probably not a good metric. Just providing some standard numbers for treatments. Keep in mind the definition of a case here is having symptoms, so we don’t know if the vaccine prevents lightly symptomatic or asymptomatic cases, which may or may not result in transmission.
- jrumbut 6y ago
  I've seen this NNT argument a few places now, and no it doesn't make any sense because most of us will be living in a society for years and years to come so if we resume normal life at some point and it's out there we all will be exposed at some point. Its current prevalence is irrelevant because if we live how we want to live it will be everywhere (assuming no vaccine).
chadash 6y ago
Key words here are "over the study duration" and "over the time period of the study". The study started in late July, so it's been about 4 months. The data that they examine probably lags a bit too and in the intervening period, cases have gone up significantly. Over the course of a longer period of time and with cases on the rise as they are now, I'd expect the risk reduction to be much higher. Edit: Also, these numbers are somewhat skewed by the fact that not all 40k people got the vaccine/placebo at once, but over a period of time.
cm2187 6y ago
Is it cases or symptomatic cases? I presume they were testing their sample regularly. Preventing symptoms would be a great outcome anyway, but it’s not the same as preventing cases.
- chadash 6y ago
  Actually, they are not. They only advise participants to get tested if they show symptoms. So it's possible that a bunch of people on the vaccine are getting asymptomatic cases.
  - marricks 6y ago
    Where did you get that info? I didn’t read it in the release. That’s not good though... really could but a kink in establishing herd immunity until everyone is actually vaccinated.
    - chadash 6y ago
      Pfizer released the study design, available at https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591.... It's a long document, but it's in there.
      - marricks 6y ago
        Thank you!
    - coldpie 6y ago
      My wife is in a different vaccine trial. For her they do a test monthly. They also do a blood draw where I would guess they are checking for antibodies, but I don't know. In addition, she reports in every single day with an app for symptom reporting.
      - virtuallynathan 6y ago
rich_sasha 6y ago
I guess you're saying - that's a lot of vaccines needed to prevent a very small number of cases - right? But presumably incidence is low because of safety restrictions (lockdowns, masks etc.) which are untenable long-term. Whereas if we can vaccinate O(everyone), we can, maybe, stop the restrictions?
- maxerickson 6y ago
  They are going to charge the US ~$40. So $800,000 to prevent 150 cases. That's incredibly cheap, and doesn't factor in an increase of social activities.
  - trebligdivad 6y ago
    if it's targeted at the elderly/at risk (at first?), the reduction in hospitalisation should be higher; and I'd think that would pay back pretty quickly.
    - maxerickson 6y ago
      Yes, it's likely a net reduction in spending, and then on top of that a bunch of harm and death is prevented. Super cheap.
- acqq 6y ago
  Already answered two days ago: https://news.ycombinator.com/item?id=25111284 https://news.ycombinator.com/item?id=25111284 In short, we still don't know which measures would have to be kept even when a lot of people are vaccinated. It will be much better known in some time, but it's still not known at this moment. It depends on sterile immunizing properties of the vaccine, which are still unknown. Edit: re the Moderna protocol mentioned in sanxiyn's comment below: even if they will measure sterilizing immunity (as one of their "Secondary" objectives), as far as I know, they still haven't published something on that topic (1) From them we still have only data on the level comparable to those from Pfizer/BioNTech: 90 cases in placebo, 5 in vaccinated, 11 severe cases, all in placebo group. Where "cases" could simply be defined "those who were recognized of having Covid-19" (as the illness). (Also see my other comment here for more general references about the primary objectives of all Phase 3 trials). Here's what Moderna only writes about their secondary objectives in the only release: "Key secondary endpoints include prevention of severe COVID-19 disease and prevention of infection by SARS-CoV-2. The trial will continue to accrue additional data relevant to safety and efficacy even after an EUA is submitted." Meaning, eventually, we should be able to know more. I didn't know that Pfizer doesn't even have that kind analysis written in their protocol, thanks. I'd hope they can add that in some later phase. 1) https://investors.modernatx.com/news-releases/news-release-details/modernas-covid-19-vaccine-candidate-meets-its-primary-efficacy https://investors.modernatx.com/news-releases/news-release-d...
y7 6y ago
Why should we care about the absolute risk reduction rather than the 20x relative risk reduction (i.e. "95% effectivity")? The idea of these fixed-term RCTs is that the numbers generalize to larger periods of time, right?
- virtuallynathan 6y ago
  I’m not sure what you mean by fixed-term. In this case, the Interim analysis was run once 170 cases occurred across both arms combined.
  - mmarvick 6y ago
    What the poster means is that going from 0.79% to 0.04% means that 0.75% were prevented from getting it due to the vaccine. You go from 79 per 10000 to 4 per 10000, meaning you saved 75 per 10000. But preventing 0.75% of the population from getting it out of the 0.79% who got it implies 94.9% (0.0075/0.0079) of those who _would_ have gotten it were prevented from getting it. That's the relative ratio. I agree with the poster that the relative number seems more interesting. There are lots of things going on right now (masks, social distancing, etc.) to prevent people from getting COVID. The fact that only 0.79% of the placebo group got it implies a lot about _other_ mitigation factors, and puts a limiting force on how many can be saved by the vaccine (a number less than or equal to 0.79%). Like OP, I wonder if there's any case where the -0.75% number is interesting in and of itself. In fact, if we weren't social distancing, presumably the prevalence of COVID would be much higher and thus the absolute ratio of people prevented from getting COVID would be higher too. The 95% effectiveness number should (theoretically) be the same no matter what other measures you're taking. It would reduce 0.79% of population from getting it to 0.04% getting it, and would reduce 10% of the population from getting it to 0.5% of the population getting it. I'm totally ignoring that lots of people getting the vaccine will lead to less people getting sick and less people spreading it (herd immunity), but you get the point.
    - Izkata 6y ago
      With absolute numbers of 162 and 8, 94.9% would have an error range of somewhere around 10%, right? So the actual percent would be somewhere in the range of 85-97%? (No actual math done here, I'm ballparking it based on looking at error ranges with sample size 100 at work; my main point is that I don't think the ~20k enrolled in each group affects this particular error range, so even though "94.9%" is stated precisely the actual data isn't nearly so precise)
tsimionescu 6y ago
I'm curious, is there any information on the confidence interval for this 95% number? Intuitively, I would expect the confidence to be pretty low - after all, if tomorrow 2 more people in the vaccine group happen to get sick, the relative risk reduction will go from 20x to 15x. Is there anything wrong with this reasoning?
- Karrot_Kream 6y ago
  No, you bring up a great question. This kind of vaccine trial is not designed to provide that information, unfortunately. The goal of this study is to show that the vaccine is at least 30% effective (defined by Vaccine Efficacy (VE) in the Pfizer protocol), with a soft goal of hitting a VE of 50%. We don't know anything about what the total population of _possible_ infections in either arm would be. That said, if we assume that there are in total 170 cases (which is true at this point in time but will change), and that there are 162 cases in the control group and 8 cases in the vaccine group, we can construct a posterior of the VE. Here's an older posterior (done with data from the press releases in the first IA a few days ago) http://blog.fellstat.com/?p=468 http://blog.fellstat.com/?p=468 that graphs this posterior. There is some uncertainty, but as you can see results are still promising. That said we just don't know how case counts will change, and while it will be fascinating to watch the results change over time, these sorts of tests are just about getting enough data to mark a vaccine as effective, not about trying to extrapolate any larger effect.
- ellisv 6y ago
  Based on publicly available numbers, the 95% Bayesian credible interval is roughly 90.5% to 97.6%.
ppf 6y ago
I hope this isn't a silly question - do we know that the placebo group represented half of the total enrolled? The Pfizer press release is very short on details.
- sanxiyn 6y ago
  Yes we do. Pfizer's trial protocol specifies 1:1 randomization. (Google "C4591001 protocol" to get it.)
nip180 6y ago
Given exponential growth of the virus, and the very large changes to society made to avoid virus population growth I think the NNT isn’t an effective measurement tool. Clearly the value of the vaccine depends heavily on where/when you are. Additionally, if the virus has an R greater than 1, preventing one person from being infected has a larger impact on total infected population.
- sanxiyn 6y ago
  That's the point. Pfizer's trial does not evaluate reduction of R or "preventing one person from being infected". It would have been better if it did, but it does not. NNT is a natural way to measure the result of this trial since the result is about prevention of disease, NOT prevention of infection.
klmadfejno 6y ago
The Moderna trial had a control group infection rate of 0.6% vs. Pfizer's 0.79%. It doesn't really change the story, but if we are going to split hairs over effectiveness ratings, I wonder how sensitive are they to random variance in the control group outcomes, assuming both drugs are exactly the same (they're not), and both tested populations are exactly the same (they're not).
cperciva 6y ago
This is a vaccine against a communicable disease. The NNT is almost certainly much lower, since preventing the 154 cases among vaccinated patients would almost certainly prevent cases among their contacts.
- acqq 6y ago
  We actually at the moment still don't know if having some people vaccinated would "prevent cases among their contacts." The published results don't show that at all. What's published is only if the vaccines prevent the occurrence of the illness (as in "did somebody have symptoms"). Also: "Animal challenge data suggest that vaccinated animals may still be infected even if they don't experience symptoms." For more details (including the whole article from which is that citation) see my other comments here.
- civilized 6y ago
  Unfortunately there is some pessimism about whether a coronavirus vaccine is likely to provide "sterilizing immunity" which would prevent spread, as opposed to just reducing the severity of the disease. https://www.statnews.com/2020/05/22/the-world-needs-covid-19-vaccines-it-may-also-be-overestimating-their-power/ https://www.statnews.com/2020/05/22/the-world-needs-covid-19...
  - ummonk 6y ago
    Would this mean we are having a bunch of asymptomatic reinfections resulting in retransmission? Surely we would have seen a lot of examples of this already with test and trace?
  - ptaipale 6y ago
    I'm a complete layman here, but I would expect that the "prevent spread" is not binary, it's a spwetrum [0,1]. Thus, even if it does not completely prevent spread, if just a part, or even a smallish part, is prevented, that also helps others. (Same thing in mask discussion. Lots of people dismiss masks because they may not completely prevent an infection; that is stupid, because masks still decrease the probability of infection (and more importantly, spreading to others).
devit 6y ago
The "absolute risk reduction" and NNT numbers seem pretty useless, since they are in a context where people are trying their best to avoid getting infected with isolation, masks, distancing, etc. Once the vaccine rolls out, people will return to previous habits and without the vaccine a large percentage of the population would be infected, so these numbers look completely different (i.e. absolute risk reduction of at least 45% from 50% to 0-5%, and NNT of something like 3-to-1 to 10-to-1).
rwoerz 6y ago
You get 20,567 in each arm??? That's a thorough study. Hope they had enough plasters. SCNR
- zeku 6y ago
  Arm in this context is a medical term referring to a study site or something like that. It's not literally someone's arm.
neilwilson 6y ago
It's not primarily about eliminating symptomatic cases. It's about eliminating excessive fear. And fear kills economic activity.
gorgoiler 6y ago
Is the study continuously rolling / ongoing?
- Karrot_Kream 6y ago
  Participants will continue to be monitored for the next 2 years. This just marks the point at which Pfizer can claim their vaccine to be more than 30% effective.
  - virtuallynathan 6y ago
    They will unfortunately vaccinate the placebo arm as soon as they get the EUA. At this point the study would be mostly unblinded, so far as I can tell, there will also be a much smaller, if any control group. “If our potential vaccine receives an EUA, we would propose amending the study protocol to create a process so that interested, eligible participants who received the placebo could ‘cross-over’ to the vaccine arm of the study,”
patall 6y ago
Note that it is 170 cases as defined in the study protocol. Which is not positive in a PCR test like you see in the media, it is positive in PCR while showing symptoms.

arijun 6y ago

Can you help interpret those numbers? Are they good?

ComputerGuru 6y ago
Really, it just means a) it works, b) the study should go on longer to get more significant results.
- timerol 6y ago
  How much longer? There is significant benefit to rolling out massive amounts of vaccine immediately. I know that any scientist worth their salt will almost always claim that more study is warranted. But we're playing against a clock measured in human lives right now
  - henrikschroder 6y ago
    And if this vaccine doesn't work or has nasty side effects, or even ends up killing people, most people will be very reluctant to take it, or any of the next better vaccines, and that would cause even more deaths. It's a hard problem, and even though the clock is ticking, we also need to get it right the first time.
- robrenaud 6y ago
  That relative risk reduction is very signficant, not just statistically signficant, it is very practically signficant too.
  - colinmhayes 6y ago
    Sure, but the sample of confirmed positives is so small that it might not include some important subgroups. If only 1 at risk person in the placebo tested positive, and 0 in the vaccinated group did we haven't learned much.
chadash 6y ago
The relevant piece for most people to know is that based on data from a short time period, people were 20x less likely to get symptomatic COVID if they get the vaccine. The exact number will probably change a bit as more time goes by and more data comes in, but 20x is the ballpark. Things we still don't know: - Does the vaccine also stop you from contracting the virus asymptomatically and spreading it? People enrolled in the trial were only told to get tested if they showed symptoms. - How long does the immunity from the vaccine last? We won't know this for some time.
- neom 6y ago
  I presume how long immunity lasts is based on how well and quickly the virus can mutate itself? Do we have any research on other Coronaviruses that might hint at how long immunity will last?
  - tsimionescu 6y ago
    That's part of it, but it also depends on the 'memory' of your immune system.
- joshyeager 6y ago
  A new study of 185 COVID patients found that 90% of the subjects were still seropositive for neutralizing antibodies at the 6 to 8 month time points. B-cells specific to the Spike and to the nucleocapsid coronavirus proteins actually increased over a five-month period. Memory T cells appeared to have half-lives of at least five or six months in these patients, and helper T cells were completely stable over the entire period studied. https://www.biorxiv.org/content/10.1101/2020.11.15.383323v1 https://www.biorxiv.org/content/10.1101/2020.11.15.383323v1

acqq 6y ago

The study enrolled more, but slowly increased the number of vaccinated since the start (which was initially, I'd expect, 0) -- the second dose was still not received by all enrolled as of November 13: "The Phase 3 clinical trial of BNT162b2 began on July 27 and has enrolled 43,661 participants to date, 41,135 of whom have received a second dose of the vaccine candidate as of November 13, 2020." So there is no single date one can reasonably use 41,135 or ~20,567 for the efficacy calculations. As the observations continue (expected up to 2 years) there could be much more useful data there. I also guess the people who haven't received both doses but got sick before are separately counted. The only useful numbers at the moment we have are 8 and 162 (and 1 and 9 for the severe cases). And that we can assume that from all thousands vaccinated these 170 in total got sick by chance, and that the thousands were carefully enough selected to represent bigger population, and using that we can expect that the efficacy won't too dramatically change as more data come in, the same way as the polls can use smaller samples to conclude something about bigger population, if the sampling is careful enough. But some changes are expected. It appears it's still too early to know if those who are vaccinated could protect those who aren't, or if the vaccine protects only those who received it (1) -- but anyway we know that still not all of those who will be vaccinated will be protected from having a severe case: there was up to now one severe case in the vaccinated group, versus 9 severe cases in the placebo group, 11%. --------- 1) https://www.medscape.com/viewarticle/941030 https://www.medscape.com/viewarticle/941030 "In an ideal world, a vaccine would prevent infection entirely and, it follows, also prevent disease and severe disease. But this may be hard to achieve for a respiratory virus vaccine. Animal challenge data suggest that vaccinated animals may still be infected even if they don't experience symptoms. A vaccine that is able to reduce the severity of disease, even if it cannot prevent infection entirely, would obviously still have enormous public health value. Therefore, this is what trials target as their primary aim."

acqq 6y ago
Maybe a better link: https://www.medscape.com/viewarticle/941030?src=soc_tw_201118_mscpedt_news_mdscp_vaccine&faf=1 https://www.medscape.com/viewarticle/941030?src=soc_tw_20111...

tgb 6y ago

Is NNT a good number to use when evaluating effectiveness of vaccines during a pandemic? It'll change substantially over time as the virus waxes and wanes and from location to location. So it won't be comparable across different vaccines. Moreover, the fact that cases create more cases means that preventing 1 case could prevent other future ones.

virtuallynathan 6y ago
I would agree, it probably not a good metric. Just providing some standard numbers for treatments. Keep in mind the definition of a case here is having symptoms, so we don’t know if the vaccine prevents lightly symptomatic or asymptomatic cases, which may or may not result in transmission.
jrumbut 6y ago
I've seen this NNT argument a few places now, and no it doesn't make any sense because most of us will be living in a society for years and years to come so if we resume normal life at some point and it's out there we all will be exposed at some point. Its current prevalence is irrelevant because if we live how we want to live it will be everywhere (assuming no vaccine).

chadash 6y ago

Key words here are "over the study duration" and "over the time period of the study". The study started in late July, so it's been about 4 months. The data that they examine probably lags a bit too and in the intervening period, cases have gone up significantly. Over the course of a longer period of time and with cases on the rise as they are now, I'd expect the risk reduction to be much higher. Edit: Also, these numbers are somewhat skewed by the fact that not all 40k people got the vaccine/placebo at once, but over a period of time.

cm2187 6y ago

Is it cases or symptomatic cases? I presume they were testing their sample regularly. Preventing symptoms would be a great outcome anyway, but it’s not the same as preventing cases.

chadash 6y ago
Actually, they are not. They only advise participants to get tested if they show symptoms. So it's possible that a bunch of people on the vaccine are getting asymptomatic cases.
- marricks 6y ago
  Where did you get that info? I didn’t read it in the release. That’s not good though... really could but a kink in establishing herd immunity until everyone is actually vaccinated.
  - chadash 6y ago
    Pfizer released the study design, available at https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591.... It's a long document, but it's in there.
    - marricks 6y ago
      Thank you!
  - coldpie 6y ago
    My wife is in a different vaccine trial. For her they do a test monthly. They also do a blood draw where I would guess they are checking for antibodies, but I don't know. In addition, she reports in every single day with an app for symptom reporting.
    - virtuallynathan 6y ago

rich_sasha 6y ago

I guess you're saying - that's a lot of vaccines needed to prevent a very small number of cases - right? But presumably incidence is low because of safety restrictions (lockdowns, masks etc.) which are untenable long-term. Whereas if we can vaccinate O(everyone), we can, maybe, stop the restrictions?

maxerickson 6y ago
They are going to charge the US ~$40. So $800,000 to prevent 150 cases. That's incredibly cheap, and doesn't factor in an increase of social activities.
- trebligdivad 6y ago
  if it's targeted at the elderly/at risk (at first?), the reduction in hospitalisation should be higher; and I'd think that would pay back pretty quickly.
  - maxerickson 6y ago
    Yes, it's likely a net reduction in spending, and then on top of that a bunch of harm and death is prevented. Super cheap.
acqq 6y ago
Already answered two days ago: https://news.ycombinator.com/item?id=25111284 https://news.ycombinator.com/item?id=25111284 In short, we still don't know which measures would have to be kept even when a lot of people are vaccinated. It will be much better known in some time, but it's still not known at this moment. It depends on sterile immunizing properties of the vaccine, which are still unknown. Edit: re the Moderna protocol mentioned in sanxiyn's comment below: even if they will measure sterilizing immunity (as one of their "Secondary" objectives), as far as I know, they still haven't published something on that topic (1) From them we still have only data on the level comparable to those from Pfizer/BioNTech: 90 cases in placebo, 5 in vaccinated, 11 severe cases, all in placebo group. Where "cases" could simply be defined "those who were recognized of having Covid-19" (as the illness). (Also see my other comment here for more general references about the primary objectives of all Phase 3 trials). Here's what Moderna only writes about their secondary objectives in the only release: "Key secondary endpoints include prevention of severe COVID-19 disease and prevention of infection by SARS-CoV-2. The trial will continue to accrue additional data relevant to safety and efficacy even after an EUA is submitted." Meaning, eventually, we should be able to know more. I didn't know that Pfizer doesn't even have that kind analysis written in their protocol, thanks. I'd hope they can add that in some later phase. 1) https://investors.modernatx.com/news-releases/news-release-details/modernas-covid-19-vaccine-candidate-meets-its-primary-efficacy https://investors.modernatx.com/news-releases/news-release-d...

y7 6y ago

Why should we care about the absolute risk reduction rather than the 20x relative risk reduction (i.e. "95% effectivity")? The idea of these fixed-term RCTs is that the numbers generalize to larger periods of time, right?

virtuallynathan 6y ago
I’m not sure what you mean by fixed-term. In this case, the Interim analysis was run once 170 cases occurred across both arms combined.
- mmarvick 6y ago
  What the poster means is that going from 0.79% to 0.04% means that 0.75% were prevented from getting it due to the vaccine. You go from 79 per 10000 to 4 per 10000, meaning you saved 75 per 10000. But preventing 0.75% of the population from getting it out of the 0.79% who got it implies 94.9% (0.0075/0.0079) of those who _would_ have gotten it were prevented from getting it. That's the relative ratio. I agree with the poster that the relative number seems more interesting. There are lots of things going on right now (masks, social distancing, etc.) to prevent people from getting COVID. The fact that only 0.79% of the placebo group got it implies a lot about _other_ mitigation factors, and puts a limiting force on how many can be saved by the vaccine (a number less than or equal to 0.79%). Like OP, I wonder if there's any case where the -0.75% number is interesting in and of itself. In fact, if we weren't social distancing, presumably the prevalence of COVID would be much higher and thus the absolute ratio of people prevented from getting COVID would be higher too. The 95% effectiveness number should (theoretically) be the same no matter what other measures you're taking. It would reduce 0.79% of population from getting it to 0.04% getting it, and would reduce 10% of the population from getting it to 0.5% of the population getting it. I'm totally ignoring that lots of people getting the vaccine will lead to less people getting sick and less people spreading it (herd immunity), but you get the point.
  - Izkata 6y ago
    With absolute numbers of 162 and 8, 94.9% would have an error range of somewhere around 10%, right? So the actual percent would be somewhere in the range of 85-97%? (No actual math done here, I'm ballparking it based on looking at error ranges with sample size 100 at work; my main point is that I don't think the ~20k enrolled in each group affects this particular error range, so even though "94.9%" is stated precisely the actual data isn't nearly so precise)

tsimionescu 6y ago

I'm curious, is there any information on the confidence interval for this 95% number? Intuitively, I would expect the confidence to be pretty low - after all, if tomorrow 2 more people in the vaccine group happen to get sick, the relative risk reduction will go from 20x to 15x. Is there anything wrong with this reasoning?

Karrot_Kream 6y ago
No, you bring up a great question. This kind of vaccine trial is not designed to provide that information, unfortunately. The goal of this study is to show that the vaccine is at least 30% effective (defined by Vaccine Efficacy (VE) in the Pfizer protocol), with a soft goal of hitting a VE of 50%. We don't know anything about what the total population of _possible_ infections in either arm would be. That said, if we assume that there are in total 170 cases (which is true at this point in time but will change), and that there are 162 cases in the control group and 8 cases in the vaccine group, we can construct a posterior of the VE. Here's an older posterior (done with data from the press releases in the first IA a few days ago) http://blog.fellstat.com/?p=468 http://blog.fellstat.com/?p=468 that graphs this posterior. There is some uncertainty, but as you can see results are still promising. That said we just don't know how case counts will change, and while it will be fascinating to watch the results change over time, these sorts of tests are just about getting enough data to mark a vaccine as effective, not about trying to extrapolate any larger effect.
ellisv 6y ago
Based on publicly available numbers, the 95% Bayesian credible interval is roughly 90.5% to 97.6%.

ppf 6y ago

I hope this isn't a silly question - do we know that the placebo group represented half of the total enrolled? The Pfizer press release is very short on details.

sanxiyn 6y ago
Yes we do. Pfizer's trial protocol specifies 1:1 randomization. (Google "C4591001 protocol" to get it.)

nip180 6y ago

Given exponential growth of the virus, and the very large changes to society made to avoid virus population growth I think the NNT isn’t an effective measurement tool. Clearly the value of the vaccine depends heavily on where/when you are. Additionally, if the virus has an R greater than 1, preventing one person from being infected has a larger impact on total infected population.

sanxiyn 6y ago
That's the point. Pfizer's trial does not evaluate reduction of R or "preventing one person from being infected". It would have been better if it did, but it does not. NNT is a natural way to measure the result of this trial since the result is about prevention of disease, NOT prevention of infection.

klmadfejno 6y ago

The Moderna trial had a control group infection rate of 0.6% vs. Pfizer's 0.79%. It doesn't really change the story, but if we are going to split hairs over effectiveness ratings, I wonder how sensitive are they to random variance in the control group outcomes, assuming both drugs are exactly the same (they're not), and both tested populations are exactly the same (they're not).

cperciva 6y ago

This is a vaccine against a communicable disease. The NNT is almost certainly much lower, since preventing the 154 cases among vaccinated patients would almost certainly prevent cases among their contacts.

acqq 6y ago
We actually at the moment still don't know if having some people vaccinated would "prevent cases among their contacts." The published results don't show that at all. What's published is only if the vaccines prevent the occurrence of the illness (as in "did somebody have symptoms"). Also: "Animal challenge data suggest that vaccinated animals may still be infected even if they don't experience symptoms." For more details (including the whole article from which is that citation) see my other comments here.
civilized 6y ago
Unfortunately there is some pessimism about whether a coronavirus vaccine is likely to provide "sterilizing immunity" which would prevent spread, as opposed to just reducing the severity of the disease. https://www.statnews.com/2020/05/22/the-world-needs-covid-19-vaccines-it-may-also-be-overestimating-their-power/ https://www.statnews.com/2020/05/22/the-world-needs-covid-19...
- ummonk 6y ago
  Would this mean we are having a bunch of asymptomatic reinfections resulting in retransmission? Surely we would have seen a lot of examples of this already with test and trace?
- ptaipale 6y ago
  I'm a complete layman here, but I would expect that the "prevent spread" is not binary, it's a spwetrum [0,1]. Thus, even if it does not completely prevent spread, if just a part, or even a smallish part, is prevented, that also helps others. (Same thing in mask discussion. Lots of people dismiss masks because they may not completely prevent an infection; that is stupid, because masks still decrease the probability of infection (and more importantly, spreading to others).

devit 6y ago

The "absolute risk reduction" and NNT numbers seem pretty useless, since they are in a context where people are trying their best to avoid getting infected with isolation, masks, distancing, etc. Once the vaccine rolls out, people will return to previous habits and without the vaccine a large percentage of the population would be infected, so these numbers look completely different (i.e. absolute risk reduction of at least 45% from 50% to 0-5%, and NNT of something like 3-to-1 to 10-to-1).

rwoerz 6y ago

You get 20,567 in each arm??? That's a thorough study. Hope they had enough plasters. SCNR

zeku 6y ago
Arm in this context is a medical term referring to a study site or something like that. It's not literally someone's arm.

neilwilson 6y ago

It's not primarily about eliminating symptomatic cases. It's about eliminating excessive fear. And fear kills economic activity.

gorgoiler 6y ago

Is the study continuously rolling / ongoing?

Karrot_Kream 6y ago
Participants will continue to be monitored for the next 2 years. This just marks the point at which Pfizer can claim their vaccine to be more than 30% effective.
- virtuallynathan 6y ago
  They will unfortunately vaccinate the placebo arm as soon as they get the EUA. At this point the study would be mostly unblinded, so far as I can tell, there will also be a much smaller, if any control group. “If our potential vaccine receives an EUA, we would propose amending the study protocol to create a process so that interested, eligible participants who received the placebo could ‘cross-over’ to the vaccine arm of the study,”

patall 6y ago

Note that it is 170 cases as defined in the study protocol. Which is not positive in a PCR test like you see in the media, it is positive in PCR while showing symptoms.