10 ms·

It looks like the largest effect found (from reading only the summary) is how many more adverse effects the Moderna placebo causes compared to the Pfizer placeb

It looks like the largest effect found (from reading only the summary) is how many more adverse effects the Moderna placebo causes compared to the Pfizer placebo. Given that, I won't regret my vaccinations just yet.

shadowgovt 4y ago
Clearly, we really need to improve quality control on the Moderna placebo. /s
smt88 4y ago
There has been decent research showing that a lot of Covid vaccine side effects appear when subjects are given placebo, which indicates that the subjects expected the side effects. So we may someday discover that vaccine misinformation causes people to experience imaginary negative effects from vaccines or placebo.
- ceejayoz 4y ago
  I'm not sure why this is downvoted; it's almost certainly the case. Many of the folks who fainted shortly after vaccine administration were probably in this category.
- goda90 4y ago
  I read too much FUD about the vaccines before my Moderna booster, particularly the myocarditis in younger men results. Started feeling tight chested and a lump in my throat later that day after the vaccine. Finally went to urgent care after a day or two and got an x-ray and EKG and it showed nothing of concern. Doctor suggested it was probably anxiety, suggested taking an NSAID and relaxing and I was better pretty soon after.
- Terretta 4y ago
  There was considerable chatter among the “vaccine hesitant” about Pfizer being “better” than Moderna, which could play into your point.
  - LorenPechtel 4y ago
    Yeah, Pfizer optimized for acceptable performance with fewer side effects, Moderna optimized for best performance with less regard for side effects. Hence the general pattern that Moderna hits you harder than Pfizer and of course the hesitant go with Pfizer if given a choice. The flip side is those of us who don't react badly will tend to go Moderna for the boosters. That also means that going forward we can't compare real world effects because the distribution will be decidedly non-random.
- origin_path 4y ago
  Please be aware that in these trials the placebo is not an inert known safe substance like water but rather another closely related vaccine. The goal is to ensure that any reactions you get are similar to the primary vaccine. So, people experiencing side effects from placebos is not really proof that they're self-created or 'expected', just that whatever vaccine was used as the placebo could also cause side effects.
- nerdponx 4y ago
  Looking at the list of adverse events is really interesting, and leads me to agree with the conclusion that this study lies somewhere between "inept bumbling" and "deliberate lying with statistics." Forget the red herring of comparing across the Pfizer and Moderna studies (which clearly have very different sample properties); the within-study comparisons are also a little suspect. Most of the adverse effects have frequencies of 1-3, which is literally the lower limit of what kind of data we can use to estimate anything about anything (ask an actuary about modeling "rare events" or "extreme events"). Moreover, what is not reported is the number of affected individuals; there is no indication about whether or not the two poor souls who ended up with pancreatitis in the Moderna trial were also the two people who ended up with diarrhea in the same trial. I am very leery of looking at the sum of these effects and concluding that, in aggregate, the Moderna vaccine is more dangerous than Covid-19 infection (let alone more dangerous than the placebo) when it comes to these "rare" adverse events -- because it doesn't make a lot of sense to aggregate these numbers. What stands out among all this rare-events noise is the risk of cardiac injury and coagulation disorder. There might actually be some meat on that bone. In the Pfizer trial, we have 8.5 instances of coagulation disorder per 10k versus 5.3 instances per 10k in the placebo. That's not nothing, and might even be "statistically significant" depending on your decision criteria. But again, without a comparison to Covid itself on this particular metric (the "risk reduction" numbers are buried in the text and not clearly enumerated), there's not much you can conclude here.
  - deleted 4y ago
    [deleted]
  - nerdponx 4y ago
    I actually went and plugged in the numbers for coagulation disorders into a frequentist likelihood ratio test for a difference in proportions, and got p = 0.23. Feel free to check my work (in R): coag_vax <- 16 p_vax <- 8.5 / 10000 num_vax <- round(coag_vax / p_vax) cat("#(vaccine) = ", num_vax, "\n", sep = "") cat("#(coagulation disorder | vaccine) = ", coag_vax, "\n", sep = "") cat("P(coagulation disorder | vaccine) = ", p_vax, "\n", sep = "") cat("\n") coag_plac <- 10 p_plac <- 5.3 / 10000 num_plac <- round(coag_plac / p_plac) cat("#(placebo) = ", num_plac, "\n", sep = "") cat("#(coagulation disorder | placebo) = ", coag_plac, "\n", sep = "") cat("P(coagulation disorder | placebo) = ", p_plac, "\n", sep = "") cat("\n") coag_same <- coag_vax + coag_plac num_same <- num_vax + num_plac p_same <- coag_same / num_same cat("#(null) = ", num_same, "\n", sep = "") cat("#(coagulation disorder | null) = ", coag_same, "\n", sep = "") cat("P(coagulation disorder | null) = ", p_same, "\n", sep = "") cat("\n") ## Frequentist likelihood ratio test that the proportions are identical # https://stats.stackexchange.com/a/373448/36229 # http://people.musc.edu/~bandyopd/bmtry711.11/lecture_02.pdf l_same <- dbinom(coag_vax, num_vax, p_same) * dbinom(coag_plac, num_plac, p_same) l_diff <- dbinom(coag_vax, num_vax, p_vax) * dbinom(coag_plac, num_plac, p_plac) t <- -2.0 * log(l_same / l_diff) p <- 1 - pchisq(t, 1) cat("LR test statistic = ", t, "\n", sep = "") cat("LR test p-value = ", p, "\n", sep = "") Output: #(vaccine) = 18824 #(coagulation disorder | vaccine) = 16 P(coagulation disorder | vaccine) = 0.00085 #(placebo) = 18868 #(coagulation disorder | placebo) = 10 P(coagulation disorder | placebo) = 0.00053 #(null) = 37692 #(coagulation disorder | null) = 26 P(coagulation disorder | null) = 0.0006898015 LR test statistic = 1.412184 LR test p-value = 0.2346942 And for a 95% CI around the difference in proportions, I got [0, 8.5]: # https://pubmed.ncbi.nlm.nih.gov/25163425/ # var_vax <- num_vax * p_vax * (1 - p_vax) # var_plac <- num_plac * p_plac * (1 - p_plac) var_pooled <- (p_vax * (1 - p_vax) / num_vax) + (p_plac * (1 - p_plac) / num_plac) std_pooled <- sqrt(var_pooled) ci_lo <- (p_vax - p_plac) - qnorm(0.975) * std_pooled ci_hi <- (p_vax - p_plac) + qnorm(0.975) * std_pooled cat("95% CI (Gaussian approximation) = [ ", max(ci_lo, 0.0) * 10000, " in 10k , ", min(ci_hi, 1.0) * 10000, " in 10k ]\n", sep = "") Output: 95% CI (Gaussian approximation) = [ 0 in 10k , 8.502481 in 10k ] I also ran it through a completely-un-tuned Bayesian model and got a similarly inconclusive result: library(brms) coag_data <- data.frame( category = c("vaccine", "placebo"), n_sample = c(num_vax, num_plac), n_coag = c(coag_vax, coag_plac)) fmla <- n_coag | trials(n_sample) ~ category prior <- get_prior(fmla, coag_data, family = binomial) cat("Prior:\n") print(prior) model <- brm( fmla, data = coag_data, family = binomial(link = "logit"), prior = prior, iter = 1000) cat("Model:\n") print(model) Output (truncated): Population-Level Effects: Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS Intercept -7.58 0.32 -8.23 -6.98 1.00 1000 1278 categoryvaccine 0.49 0.41 -0.27 1.32 1.00 1319 1239 Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS and Tail_ESS are effective sample size measures, and Rhat is the potential scale reduction factor on split chains (at convergence, Rhat = 1). Conclusion: It's suggestive that the vaccine is more dangerous than the placebo but not at all a strong confirmation. I did skim back over the paper and it looks like they categorized "serious" in such a way that maybe hospitalization is not a terrible baseline for comparison. But it's still pretty questionable that the incidence of "serious" coagulation disorders in the placebo group is higher than the incidence of hospitalization in the placebo group, and it makes me think that they're making an invalid comparison there. So this is an inconclusive result, presented in what is at best a hand-wavey framework.
everforward 4y ago
Someone pointed out above, those numbers are from 2 different trials, and they counted placebos and SAEs differently. The two trials are internally consistent, but can't be compared against each other due to differences in procedure and methodology.

shadowgovt 4y ago

Clearly, we really need to improve quality control on the Moderna placebo. /s

smt88 4y ago

There has been decent research showing that a lot of Covid vaccine side effects appear when subjects are given placebo, which indicates that the subjects expected the side effects. So we may someday discover that vaccine misinformation causes people to experience imaginary negative effects from vaccines or placebo.

ceejayoz 4y ago
I'm not sure why this is downvoted; it's almost certainly the case. Many of the folks who fainted shortly after vaccine administration were probably in this category.
goda90 4y ago
I read too much FUD about the vaccines before my Moderna booster, particularly the myocarditis in younger men results. Started feeling tight chested and a lump in my throat later that day after the vaccine. Finally went to urgent care after a day or two and got an x-ray and EKG and it showed nothing of concern. Doctor suggested it was probably anxiety, suggested taking an NSAID and relaxing and I was better pretty soon after.
Terretta 4y ago
There was considerable chatter among the “vaccine hesitant” about Pfizer being “better” than Moderna, which could play into your point.
- LorenPechtel 4y ago
  Yeah, Pfizer optimized for acceptable performance with fewer side effects, Moderna optimized for best performance with less regard for side effects. Hence the general pattern that Moderna hits you harder than Pfizer and of course the hesitant go with Pfizer if given a choice. The flip side is those of us who don't react badly will tend to go Moderna for the boosters. That also means that going forward we can't compare real world effects because the distribution will be decidedly non-random.
origin_path 4y ago
Please be aware that in these trials the placebo is not an inert known safe substance like water but rather another closely related vaccine. The goal is to ensure that any reactions you get are similar to the primary vaccine. So, people experiencing side effects from placebos is not really proof that they're self-created or 'expected', just that whatever vaccine was used as the placebo could also cause side effects.
nerdponx 4y ago
Looking at the list of adverse events is really interesting, and leads me to agree with the conclusion that this study lies somewhere between "inept bumbling" and "deliberate lying with statistics." Forget the red herring of comparing across the Pfizer and Moderna studies (which clearly have very different sample properties); the within-study comparisons are also a little suspect. Most of the adverse effects have frequencies of 1-3, which is literally the lower limit of what kind of data we can use to estimate anything about anything (ask an actuary about modeling "rare events" or "extreme events"). Moreover, what is not reported is the number of affected individuals; there is no indication about whether or not the two poor souls who ended up with pancreatitis in the Moderna trial were also the two people who ended up with diarrhea in the same trial. I am very leery of looking at the sum of these effects and concluding that, in aggregate, the Moderna vaccine is more dangerous than Covid-19 infection (let alone more dangerous than the placebo) when it comes to these "rare" adverse events -- because it doesn't make a lot of sense to aggregate these numbers. What stands out among all this rare-events noise is the risk of cardiac injury and coagulation disorder. There might actually be some meat on that bone. In the Pfizer trial, we have 8.5 instances of coagulation disorder per 10k versus 5.3 instances per 10k in the placebo. That's not nothing, and might even be "statistically significant" depending on your decision criteria. But again, without a comparison to Covid itself on this particular metric (the "risk reduction" numbers are buried in the text and not clearly enumerated), there's not much you can conclude here.
- deleted 4y ago
  [deleted]
- nerdponx 4y ago
  I actually went and plugged in the numbers for coagulation disorders into a frequentist likelihood ratio test for a difference in proportions, and got p = 0.23. Feel free to check my work (in R): coag_vax <- 16 p_vax <- 8.5 / 10000 num_vax <- round(coag_vax / p_vax) cat("#(vaccine) = ", num_vax, "\n", sep = "") cat("#(coagulation disorder | vaccine) = ", coag_vax, "\n", sep = "") cat("P(coagulation disorder | vaccine) = ", p_vax, "\n", sep = "") cat("\n") coag_plac <- 10 p_plac <- 5.3 / 10000 num_plac <- round(coag_plac / p_plac) cat("#(placebo) = ", num_plac, "\n", sep = "") cat("#(coagulation disorder | placebo) = ", coag_plac, "\n", sep = "") cat("P(coagulation disorder | placebo) = ", p_plac, "\n", sep = "") cat("\n") coag_same <- coag_vax + coag_plac num_same <- num_vax + num_plac p_same <- coag_same / num_same cat("#(null) = ", num_same, "\n", sep = "") cat("#(coagulation disorder | null) = ", coag_same, "\n", sep = "") cat("P(coagulation disorder | null) = ", p_same, "\n", sep = "") cat("\n") ## Frequentist likelihood ratio test that the proportions are identical # https://stats.stackexchange.com/a/373448/36229 # http://people.musc.edu/~bandyopd/bmtry711.11/lecture_02.pdf l_same <- dbinom(coag_vax, num_vax, p_same) * dbinom(coag_plac, num_plac, p_same) l_diff <- dbinom(coag_vax, num_vax, p_vax) * dbinom(coag_plac, num_plac, p_plac) t <- -2.0 * log(l_same / l_diff) p <- 1 - pchisq(t, 1) cat("LR test statistic = ", t, "\n", sep = "") cat("LR test p-value = ", p, "\n", sep = "") Output: #(vaccine) = 18824 #(coagulation disorder | vaccine) = 16 P(coagulation disorder | vaccine) = 0.00085 #(placebo) = 18868 #(coagulation disorder | placebo) = 10 P(coagulation disorder | placebo) = 0.00053 #(null) = 37692 #(coagulation disorder | null) = 26 P(coagulation disorder | null) = 0.0006898015 LR test statistic = 1.412184 LR test p-value = 0.2346942 And for a 95% CI around the difference in proportions, I got [0, 8.5]: # https://pubmed.ncbi.nlm.nih.gov/25163425/ # var_vax <- num_vax * p_vax * (1 - p_vax) # var_plac <- num_plac * p_plac * (1 - p_plac) var_pooled <- (p_vax * (1 - p_vax) / num_vax) + (p_plac * (1 - p_plac) / num_plac) std_pooled <- sqrt(var_pooled) ci_lo <- (p_vax - p_plac) - qnorm(0.975) * std_pooled ci_hi <- (p_vax - p_plac) + qnorm(0.975) * std_pooled cat("95% CI (Gaussian approximation) = [ ", max(ci_lo, 0.0) * 10000, " in 10k , ", min(ci_hi, 1.0) * 10000, " in 10k ]\n", sep = "") Output: 95% CI (Gaussian approximation) = [ 0 in 10k , 8.502481 in 10k ] I also ran it through a completely-un-tuned Bayesian model and got a similarly inconclusive result: library(brms) coag_data <- data.frame( category = c("vaccine", "placebo"), n_sample = c(num_vax, num_plac), n_coag = c(coag_vax, coag_plac)) fmla <- n_coag | trials(n_sample) ~ category prior <- get_prior(fmla, coag_data, family = binomial) cat("Prior:\n") print(prior) model <- brm( fmla, data = coag_data, family = binomial(link = "logit"), prior = prior, iter = 1000) cat("Model:\n") print(model) Output (truncated): Population-Level Effects: Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS Intercept -7.58 0.32 -8.23 -6.98 1.00 1000 1278 categoryvaccine 0.49 0.41 -0.27 1.32 1.00 1319 1239 Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS and Tail_ESS are effective sample size measures, and Rhat is the potential scale reduction factor on split chains (at convergence, Rhat = 1). Conclusion: It's suggestive that the vaccine is more dangerous than the placebo but not at all a strong confirmation. I did skim back over the paper and it looks like they categorized "serious" in such a way that maybe hospitalization is not a terrible baseline for comparison. But it's still pretty questionable that the incidence of "serious" coagulation disorders in the placebo group is higher than the incidence of hospitalization in the placebo group, and it makes me think that they're making an invalid comparison there. So this is an inconclusive result, presented in what is at best a hand-wavey framework.

everforward 4y ago

Someone pointed out above, those numbers are from 2 different trials, and they counted placebos and SAEs differently. The two trials are internally consistent, but can't be compared against each other due to differences in procedure and methodology.