5 ms·

IANAD but I'm nearly positive you're incorrect. I spent last night puzzling over what "efficacy" means vis a vis this video: https://www.technologynetworks.com

IANAD but I'm nearly positive you're incorrect. I spent last night puzzling over what "efficacy" means vis a vis this video: https://www.technologynetworks.com/immunology/videos/covid-19-vaccine-efficacy-explained-its-not-what-you-think-342956 https://www.technologynetworks.com/immunology/videos/covid-1... The author explains that the efficacy number is derived from the the number of the number of people in the control/placebo group infected divided by the total number of infections in the study. 76% efficacy means that, for a given population of people who contracted a disease (ignoring all uninfected), 24% were vaccinated (vaccine failures), 76% were not vaccinated. It has nothing to do with reduction of symptoms. I think this is unintuitive because the "real" solution that laypersons want is, what percentage of vaccinated people exposed to a disease get infected? We can't measure that directly because we can't just expose a bunch of folks to Covid and hope for the best. But if the sample of placebo and vaccinated people is truly random, this would appear to arrive at the same ratio. (re: when I said "ignoring all uninfected"? in a good sample, the ratio of unexposed vaccinated and control units should be about the same)

renewiltord 6y ago
> We can't measure that directly because we can't just expose a bunch of folks to Covid and hope for the best. We can, but medical ethics hasn't caught up yet. I would participate in a challenge trial, for instance.
- jMyles 6y ago
  > I would participate in a challenge trial, for instance. Me too. And nearly everyone I know in my risk tier. We had an entire globe of low-risk people willing to be part of the most acute spread, whether for research, for personal immunity, or to be able to isolate for a defined time to protect loved ones, and we squandered it in favor of pseudoscientific horizontal interdictions like lockdowns. It's really an incredible abandonment of basic principles of public health.
  - renewiltord 6y ago
    Medical ethicists believe they've solved the trolley problem by turning their back on the lever and letting Jesus take the wheel.
  - carschno 6y ago
    If the sample consists of volunteers, it's hardly random. In particular, given that these volunteers are "low-risk people", it means that the results of this experiment will yield virtually no useful conclusions for the higher-risk groups
    - jMyles 6y ago
      Sure, I don't mean to suggest that it can be a replacement for RCTs for vaccines. But there's a lot of information we left on the table. We had an opportunity to systematically formulate a controlled acute spread, studying the virology in a longitudinal way, studying the immune response in a more intimate and controlled way (from antibodies to the immune response after antibodies fade), and to examine the prevalence and role of innate immunity. Beyond research, we had a chance to have tens of millions of people with verifiable immunity in Spring or Summer. People who then were in a great position to render aid to the marginalized communities who are instead bearing the brunt of lockdowns. Instead, we traded all that for zero upside. And we have to deal with the enormous side effects of the horizontal interdiction as well.
    - zensavona 6y ago
      Exactly, this is the real issue here unfortunately.
- throwaway2245 6y ago
  A randomly controlled challenge trial would involve deliberately infecting scores of unvaccinated people, putting them at risk, along with anyone involved in the procedure and anyone they come into contact. It would mean handling live virus, which requires a Biosafety Level 3 facilities. And, it's not going to get your vaccine approved any quicker.
  - renewiltord 6y ago
    See, this is a classic example. You need BSL3 facilities if you want to have a hundred people have it. But the alternative is 27 million people getting it in planes, boats, and buses - none of which are BSL3. That part is okay. Quite the example of the asymmetry. If you engage with the problem you have to operate absolutely perfectly. Way better not to engage with it and kill a few hundred thousand people. After all, no one can blame you for that part. And of course it's not going to get the vaccine 'approved' quicker because yeah, 'approval' is also subject to the same people. But it will let us know if it works and whether it hurts.
    - throwaway2245 6y ago
      > You need BSL3 facilities if you want to have a hundred people have it. But the alternative is 27 million people getting it in planes, boats, and buses - none of which are BSL3. Those things are not true alternatives. I said "it's not going to get your vaccine approved any quicker" in the hope that you would understand: vaccines have reached the point of global distribution without doing the additional deliberate harm of intentionally infecting people.
      - renewiltord 6y ago
        Well, we've taken your approach and killed 300k in the process, so I guess we could play at being tankies in the '90s and say "If only the Communism were properly done" or we could revisit the approach. Of course I am healthy, my family has had the vaccine or the disease and punched through, so I don't really care all that much.
      - 6y ago
elliekelly 6y ago
That was a super interesting video, thanks for sharing!
hutzlibu 6y ago
It is so frustrating, trying to figure out at least a bit of whats going on and understanding some numbers, but apparently I cannot do this, without having to dive into papers, too? There is so much confusion and missinformation flying around, when even here on HN there is discussion about the meaning of basic numbers. (but please carry on discussing it)
- akiselev 6y ago
  To be clear, this isn't some settled math like basic algebra or science like Newtonian physics - this is cutting edge stuff at the intersection of medicine, public health, and biotechnology. Experts disagree about the nuances of measuring the efficacy of drugs, vaccines, and therapeutics so throw in the glaring ethical concerns into the mix and you have a recipe for a complex field. It's rare (unheard of?) to find a clinical trial where the company (or their CRO) scientists didn't sit down with FDA regulators and negotiate over key features & risk indicators, benchmarking, patient selection, and so on because they bring expertise the FDA is unlikely to have. It's very difficult to get even a basic grasp of the details because you need to grok some complex statistics, biology, and public health to really dig in. Journalists sure aren't going to get the nuances so of course you have to read the academic literature.
  - hutzlibu 6y ago
    I get it, that I won't really get it, without studying it. But it would be nice, to know what important numbers roughly mean, without that amount of work.
StavrosK 6y ago
> The author explains that the efficacy number is derived from the the number of the number of people in the control/placebo group infected divided by the total number of infections in the study. Wouldn't that make a vaccine that does nothing 50% effective, instead of 0% effective (which is what one would expect)?
- Eric_WVGG 6y ago
  Well, let's see… say you had 2000 participants in the trial, 1000 with a junk vaccine, and 1000 with placebos. Furthermore, let's say you had 200 infections, split equally between the junk and the placebo. So yeah, 100 / 200 = 50% effective. A coin toss, which is to say, no better or worse than not getting the vaccine. Again, this isn't measuring "what one would expect" (how many vaccinated people, exposed to the virus, get the disease), it's "what proportion of vaccinated people get sick relative to unvaccinated people." (again IANAD)
  - StavrosK 6y ago
    Kind of seems odd, given that you can never have 0% efficacy (unless your vaccine somehow caused every vaccinated person to get the disease instead of a non-vaccinated person). It seems like you're wasting half your "spread", basically. If you measured efficacy as "1 - (vaccinated infections / unvaccinated infections)", it would be much more intuitive, and you could even get a bit of negative efficacy in the case where your vaccine did nothing.
  - iggldiggl 6y ago
    No, 50 % means the infection rate (or whatever you're actually measuring) in the vaccine group is half that of the control group. In your example the infection rate are exactly the same, giving an efficacy of 0 % (± whatever confidence interval the size of your trial and placebo groups gives you). See e.g. https://blogs.sciencemag.org/pipeline/archives/2020/09/21/the-vaccine-protocols https://blogs.sciencemag.org/pipeline/archives/2020/09/21/th...
deleted 6y ago
[deleted]

renewiltord 6y ago

> We can't measure that directly because we can't just expose a bunch of folks to Covid and hope for the best. We can, but medical ethics hasn't caught up yet. I would participate in a challenge trial, for instance.

jMyles 6y ago
> I would participate in a challenge trial, for instance. Me too. And nearly everyone I know in my risk tier. We had an entire globe of low-risk people willing to be part of the most acute spread, whether for research, for personal immunity, or to be able to isolate for a defined time to protect loved ones, and we squandered it in favor of pseudoscientific horizontal interdictions like lockdowns. It's really an incredible abandonment of basic principles of public health.
- renewiltord 6y ago
  Medical ethicists believe they've solved the trolley problem by turning their back on the lever and letting Jesus take the wheel.
- carschno 6y ago
  If the sample consists of volunteers, it's hardly random. In particular, given that these volunteers are "low-risk people", it means that the results of this experiment will yield virtually no useful conclusions for the higher-risk groups
  - jMyles 6y ago
    Sure, I don't mean to suggest that it can be a replacement for RCTs for vaccines. But there's a lot of information we left on the table. We had an opportunity to systematically formulate a controlled acute spread, studying the virology in a longitudinal way, studying the immune response in a more intimate and controlled way (from antibodies to the immune response after antibodies fade), and to examine the prevalence and role of innate immunity. Beyond research, we had a chance to have tens of millions of people with verifiable immunity in Spring or Summer. People who then were in a great position to render aid to the marginalized communities who are instead bearing the brunt of lockdowns. Instead, we traded all that for zero upside. And we have to deal with the enormous side effects of the horizontal interdiction as well.
  - zensavona 6y ago
    Exactly, this is the real issue here unfortunately.
throwaway2245 6y ago
A randomly controlled challenge trial would involve deliberately infecting scores of unvaccinated people, putting them at risk, along with anyone involved in the procedure and anyone they come into contact. It would mean handling live virus, which requires a Biosafety Level 3 facilities. And, it's not going to get your vaccine approved any quicker.
- renewiltord 6y ago
  See, this is a classic example. You need BSL3 facilities if you want to have a hundred people have it. But the alternative is 27 million people getting it in planes, boats, and buses - none of which are BSL3. That part is okay. Quite the example of the asymmetry. If you engage with the problem you have to operate absolutely perfectly. Way better not to engage with it and kill a few hundred thousand people. After all, no one can blame you for that part. And of course it's not going to get the vaccine 'approved' quicker because yeah, 'approval' is also subject to the same people. But it will let us know if it works and whether it hurts.
  - throwaway2245 6y ago
    > You need BSL3 facilities if you want to have a hundred people have it. But the alternative is 27 million people getting it in planes, boats, and buses - none of which are BSL3. Those things are not true alternatives. I said "it's not going to get your vaccine approved any quicker" in the hope that you would understand: vaccines have reached the point of global distribution without doing the additional deliberate harm of intentionally infecting people.
    - renewiltord 6y ago
      Well, we've taken your approach and killed 300k in the process, so I guess we could play at being tankies in the '90s and say "If only the Communism were properly done" or we could revisit the approach. Of course I am healthy, my family has had the vaccine or the disease and punched through, so I don't really care all that much.
    - 6y ago

elliekelly 6y ago

That was a super interesting video, thanks for sharing!

hutzlibu 6y ago

It is so frustrating, trying to figure out at least a bit of whats going on and understanding some numbers, but apparently I cannot do this, without having to dive into papers, too? There is so much confusion and missinformation flying around, when even here on HN there is discussion about the meaning of basic numbers. (but please carry on discussing it)

akiselev 6y ago
To be clear, this isn't some settled math like basic algebra or science like Newtonian physics - this is cutting edge stuff at the intersection of medicine, public health, and biotechnology. Experts disagree about the nuances of measuring the efficacy of drugs, vaccines, and therapeutics so throw in the glaring ethical concerns into the mix and you have a recipe for a complex field. It's rare (unheard of?) to find a clinical trial where the company (or their CRO) scientists didn't sit down with FDA regulators and negotiate over key features & risk indicators, benchmarking, patient selection, and so on because they bring expertise the FDA is unlikely to have. It's very difficult to get even a basic grasp of the details because you need to grok some complex statistics, biology, and public health to really dig in. Journalists sure aren't going to get the nuances so of course you have to read the academic literature.
- hutzlibu 6y ago
  I get it, that I won't really get it, without studying it. But it would be nice, to know what important numbers roughly mean, without that amount of work.

StavrosK 6y ago

> The author explains that the efficacy number is derived from the the number of the number of people in the control/placebo group infected divided by the total number of infections in the study. Wouldn't that make a vaccine that does nothing 50% effective, instead of 0% effective (which is what one would expect)?

Eric_WVGG 6y ago
Well, let's see… say you had 2000 participants in the trial, 1000 with a junk vaccine, and 1000 with placebos. Furthermore, let's say you had 200 infections, split equally between the junk and the placebo. So yeah, 100 / 200 = 50% effective. A coin toss, which is to say, no better or worse than not getting the vaccine. Again, this isn't measuring "what one would expect" (how many vaccinated people, exposed to the virus, get the disease), it's "what proportion of vaccinated people get sick relative to unvaccinated people." (again IANAD)
- StavrosK 6y ago
  Kind of seems odd, given that you can never have 0% efficacy (unless your vaccine somehow caused every vaccinated person to get the disease instead of a non-vaccinated person). It seems like you're wasting half your "spread", basically. If you measured efficacy as "1 - (vaccinated infections / unvaccinated infections)", it would be much more intuitive, and you could even get a bit of negative efficacy in the case where your vaccine did nothing.
- iggldiggl 6y ago
  No, 50 % means the infection rate (or whatever you're actually measuring) in the vaccine group is half that of the control group. In your example the infection rate are exactly the same, giving an efficacy of 0 % (± whatever confidence interval the size of your trial and placebo groups gives you). See e.g. https://blogs.sciencemag.org/pipeline/archives/2020/09/21/the-vaccine-protocols https://blogs.sciencemag.org/pipeline/archives/2020/09/21/th...

deleted 6y ago

[deleted]