4 ms·

> One of the most damaging moments was when French President Emmanuel Macron openly disparaged the jab in January, calling it quasi-ineffective. What do you wa

> One of the most damaging moments was when French President Emmanuel Macron openly disparaged the jab in January, calling it quasi-ineffective. What do you want people to say when you report a 6.3% efficacy (against serious illness in over 65's) in a phase 3 clinical trial? Sure it was a small sample and thus statistically useless blah blah, but design your trials better.

alimw 5y ago
I never heard of a 6.3% figure for efficacy in any sample, small or otherwise. Are you sure that's what your figure refers to?
- shapefrog 5y ago
  That is the efficacy results from the AZ phase 3 clinical trial, in the over 65 group as published by AZ. I can sense your doubt ... we can even go to the factcheckers to see it. https://fullfact.org/health/german-astrazeneca-8-percent-handelsblatt/ https://fullfact.org/health/german-astrazeneca-8-percent-han... Robert Koch Institute (RKI), including its calculations on the vaccine’s efficacy data submitted by the manufacturers. These appeared to show the vaccine was just 6.3% effective among over 65s.
  - yokaze 5y ago
    The next sentence right after the one you chose to quote in your last paragraph: "This is close to 8%. However this number shouldn’t be taken at face value." Why? TLDR: Too small group and too few cases in groups (one each) to be statistical significant.
    - shapefrog 5y ago
      Yeah the 8% was circulating in the media as the efficacy and thus this was debunking it as not the efficacy. I said at the start not statistically significant. That is becuase they designed a shitty trial, if you are going to have an insignificant number of 65+'s in the trial, then dont make the target endpoint 'the number of cases' then when reporting a low efficacy are shocked when people took it on face value and ignored the p-value. If you want to test for side effects / will it kill people then make that the endpoint for that section of the trial. That is horrifically bad science and communication. But if you publish a headline number of 70% efficacy in under 65's and 6.3% efficacy in over 65's what are you meant to think? That is might be quasi-ineffective in over 65s?
  - alimw 5y ago
    Thanks! I don't speak German but it looks like there were 341 who took the vaccine, 319 who took the placebo and 1 who got sick in each of those groups. Solve for e in (1 - e) * (1 / 319) = 1 /341 and you get 6.5%. Almost...
- fabian2k 5y ago
  If I remember this correctly, that age group had so few cases that the error bars essentially covered most of the entire possible range. So even if you have a value like 6.3% this might actually be something like 6.3% +- 60%. And it that case the single value is extremely misleading, the true result is "we don't know". I'm not sure I'm remembering the right case here, this happened more than once e.g. with some data from South Africa where low values were quotes without explaining that the studies simply were not large enough to get useful numbers for these subgroups.

alimw 5y ago

I never heard of a 6.3% figure for efficacy in any sample, small or otherwise. Are you sure that's what your figure refers to?

shapefrog 5y ago
That is the efficacy results from the AZ phase 3 clinical trial, in the over 65 group as published by AZ. I can sense your doubt ... we can even go to the factcheckers to see it. https://fullfact.org/health/german-astrazeneca-8-percent-handelsblatt/ https://fullfact.org/health/german-astrazeneca-8-percent-han... Robert Koch Institute (RKI), including its calculations on the vaccine’s efficacy data submitted by the manufacturers. These appeared to show the vaccine was just 6.3% effective among over 65s.
- yokaze 5y ago
  The next sentence right after the one you chose to quote in your last paragraph: "This is close to 8%. However this number shouldn’t be taken at face value." Why? TLDR: Too small group and too few cases in groups (one each) to be statistical significant.
  - shapefrog 5y ago
    Yeah the 8% was circulating in the media as the efficacy and thus this was debunking it as not the efficacy. I said at the start not statistically significant. That is becuase they designed a shitty trial, if you are going to have an insignificant number of 65+'s in the trial, then dont make the target endpoint 'the number of cases' then when reporting a low efficacy are shocked when people took it on face value and ignored the p-value. If you want to test for side effects / will it kill people then make that the endpoint for that section of the trial. That is horrifically bad science and communication. But if you publish a headline number of 70% efficacy in under 65's and 6.3% efficacy in over 65's what are you meant to think? That is might be quasi-ineffective in over 65s?
- alimw 5y ago
  Thanks! I don't speak German but it looks like there were 341 who took the vaccine, 319 who took the placebo and 1 who got sick in each of those groups. Solve for e in (1 - e) * (1 / 319) = 1 /341 and you get 6.5%. Almost...
fabian2k 5y ago
If I remember this correctly, that age group had so few cases that the error bars essentially covered most of the entire possible range. So even if you have a value like 6.3% this might actually be something like 6.3% +- 60%. And it that case the single value is extremely misleading, the true result is "we don't know". I'm not sure I'm remembering the right case here, this happened more than once e.g. with some data from South Africa where low values were quotes without explaining that the studies simply were not large enough to get useful numbers for these subgroups.