5 ms·

When reading this sentence: > We did not observe an increased incidence of neither pericarditis nor myocarditis in adult patients recovering from COVID-19 infe

When reading this sentence: > We did not observe an increased incidence of neither pericarditis nor myocarditis in adult patients recovering from COVID-19 infection. you should be careful not to interpret this as evidence of absence. The previous sentence gives confidence intervals: > Post COVID-19 infection was not associated with either myocarditis (aHR 1.08; 95% CI 0.45 to 2.56) or pericarditis (aHR 0.53; 95% CI 0.25 to 1.13). Here a 95% CI from 0.45 to 2.56 means that the data is statistically consistent with myocarditis being 0.45 times as likely in people with COVID infection, or 2.56 times more likely. The sample size here is simply not enough to be more precise, simply because myocarditis and pericarditis are so rare. (It's hard to be precise about the rate of something that happened in only nine COVID patients in the study!) Hence you should not interpret this as demonstrating no difference between people with and without prior COVID infection -- merely bounding the size of the difference that may exist, and saying it can't be more than roughly a factor of 2.5-3. To be blunt, the authors are wrong to say in their Conclusion that "Our data suggest that there is no increase in the incidence of myocarditis and pericarditis in COVID-19 recovered patients compared to uninfected matched controls." Again, the confidence interval shows that a 2x increased incidence of myocarditis would be consistent with the data. This is a widespread misinterpretation of statistics and one that has been around for decades, so please excuse me for taking every opportunity to complain. (e.g. https://www.statisticsdonewrong.com/power.html#the-power-of-being-underpowered https://www.statisticsdonewrong.com/power.html#the-power-of-...)

jimcavel888 4y ago
clairity 4y ago
right, i was going to make a similar kind of critique (but probably less lucidly). the effect sizes are tiny and the confidence intervals overwhelm the supposed conclusions. seems to be motivated reasoning rather than impartial science. covid may cause a rise in heart disease in unvaxed individuals relative to vaxed, but this study does nothing to support, and especially doesn't prove, that contention. one thing it does reveal is that the overall incidence rates for myo-/pericarditis are pretty small, vaxed or otherwise. what would be more interesting is a comparative study of incidence relative to similar cardiopulmonary infections like the flu. that would tell us whether heart disease from covid is something to worry more about than our ambient worry over heart disease in general (which seems relatively mild compared to covid).
- chiefalchemist 4y ago
  > Covid may cause a rise in heart disease in unvaxed individuals relative to vaxed, but this study does nothing to support, and especially doesn't prove, that contention. The issue that comes to mind for me is there no explanation given on why the unvaccinated were unvaccinated. From what I remember, the jab was not recommended for those with compromised immune system or other similar less-than-healthy conditions. Without such transparency, it feels like possible apple to orangers comparisons. Or maybe I'm misreading?
  - clairity 4y ago
    that could be a confounding factor, but it's hard to know for sure without really digging into their data and methods. they may have tried to correct for that. in any case, this study actually seems to suggest that while vaccination probably has a marginal benefit, the effect size is too small to really matter.
  - random314 4y ago
    A double dose is recommended for the immunocompromised
antipaul 4y ago
The confidence interval is a so-called random variable, which means, given it’s not significant, you actually can’t really interpret it Descriptively, there is a trend, yes. But it can not be interpreted in statistical (inferential) terms, I think.
- manwe150 4y ago
  Right, the parent post was making the same point: the article claims to have shown a lack of statistical correlation, but all they did was show that their study lacked the ability to detect correlation. It is a subtle movement of a few words, but they are important It can be interpreted in statistical terms. They could use the confidence interval instead to state that the study suggests infection reduces these adverse events by no more than half, nor increases their frequency by no more than triple.
onos 4y ago
You’ve only emphasized one side of the interval, which suggests some bias. Eg you haven’t written out that the other side implies you might have a smaller chance of these issues once you get covid. That said, it probably is reasonable to have as a prior that it’s unlikely that covid would lower your risk of these issues.
- brookst 4y ago
  Did they not emphasize both sides of the uncertainty? > the data is statistically consistent with myocarditis being 0.45 times as likely in people with COVID infection, or 2.56 times more likely.
  - manwe150 4y ago
    That feels an awkward way to word that claim, since both of those outcomes have mathematically zero chance of being true literally (interpreted as points), or less than 5% total chance of being true (interpreted as areas) I think the converse statement would have been more strictly true statistically (e.g. that 0.45 is statistically consistent with the data), and more accurately descriptive of what a p-test actually means.
LennyHenrysNuts 4y ago
So would this study have been able to show a massive uptick in myocarditis/pericarditis (if it had occurred)? Or is it pretty much useless? From my limited understanding, 2.56x next to none is still pretty close to next to none.
- capnrefsmmat 4y ago
  It would presumably be able to detect a massive increase, yes, if by "massive" you mean something like a 10x increase in rate. One could do a power calculation to determine its ability to detect different sizes of effect. Normally you'd do that when planning a study to determine the sample you need, but here they took every available patient in the medical records database they had access to, so they didn't really have a choice of sample size.
amluto 4y ago
I beg to differ. I think you and the paper are committing another falacy: getting hung up on relative instead of absolute hazards. Myocarditis and pericarditis (at least the reportable kind studied in the paper) are rare — all groups had incidence well under 0.01%. That’s 10 per 100k for those who prefer those units. The study enrolled almost 200k COVID patients and over 500k controls. This is a lot of power to bound the absolute risk, but it is indeed underpowered to get good bounds on the hazard ratio. The paper would be substantially improved if it said “the absolute risk of myo/pericarditis in COVID patients is <10 per 100k with confidence 99.9%” or “the increased risk is <9 per 100k with confidence such-and-such” (I made up the numbers, but the analysis is easy.) So yes, a 2x increased risk would be consistent with the data, but an actual substantial risk would not be consistent with the data.
- capnrefsmmat 4y ago
  Yes, I agree that it's important to put the absolute risk in context. The NEJM paper that started the concern about myocarditis put it this way: > Vaccination was most strongly associated with an elevated risk of myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9025013/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9025013/ Giving the risk difference in terms of events per 100,000 people is useful. It's also trickier to interpret because the length of the follow-up period is important: if the vaccine changes the rate of events, then a study that observes patients for two years after vaccination will have (roughly) double the number of events of a study that follows patients for one year after vaccination. That's why people like relative risks in this setting.

jimcavel888 4y ago

clairity 4y ago

right, i was going to make a similar kind of critique (but probably less lucidly). the effect sizes are tiny and the confidence intervals overwhelm the supposed conclusions. seems to be motivated reasoning rather than impartial science. covid may cause a rise in heart disease in unvaxed individuals relative to vaxed, but this study does nothing to support, and especially doesn't prove, that contention. one thing it does reveal is that the overall incidence rates for myo-/pericarditis are pretty small, vaxed or otherwise. what would be more interesting is a comparative study of incidence relative to similar cardiopulmonary infections like the flu. that would tell us whether heart disease from covid is something to worry more about than our ambient worry over heart disease in general (which seems relatively mild compared to covid).

chiefalchemist 4y ago
> Covid may cause a rise in heart disease in unvaxed individuals relative to vaxed, but this study does nothing to support, and especially doesn't prove, that contention. The issue that comes to mind for me is there no explanation given on why the unvaccinated were unvaccinated. From what I remember, the jab was not recommended for those with compromised immune system or other similar less-than-healthy conditions. Without such transparency, it feels like possible apple to orangers comparisons. Or maybe I'm misreading?
- clairity 4y ago
  that could be a confounding factor, but it's hard to know for sure without really digging into their data and methods. they may have tried to correct for that. in any case, this study actually seems to suggest that while vaccination probably has a marginal benefit, the effect size is too small to really matter.
- random314 4y ago
  A double dose is recommended for the immunocompromised

antipaul 4y ago

The confidence interval is a so-called random variable, which means, given it’s not significant, you actually can’t really interpret it Descriptively, there is a trend, yes. But it can not be interpreted in statistical (inferential) terms, I think.

manwe150 4y ago
Right, the parent post was making the same point: the article claims to have shown a lack of statistical correlation, but all they did was show that their study lacked the ability to detect correlation. It is a subtle movement of a few words, but they are important It can be interpreted in statistical terms. They could use the confidence interval instead to state that the study suggests infection reduces these adverse events by no more than half, nor increases their frequency by no more than triple.

onos 4y ago

You’ve only emphasized one side of the interval, which suggests some bias. Eg you haven’t written out that the other side implies you might have a smaller chance of these issues once you get covid. That said, it probably is reasonable to have as a prior that it’s unlikely that covid would lower your risk of these issues.

brookst 4y ago
Did they not emphasize both sides of the uncertainty? > the data is statistically consistent with myocarditis being 0.45 times as likely in people with COVID infection, or 2.56 times more likely.
- manwe150 4y ago
  That feels an awkward way to word that claim, since both of those outcomes have mathematically zero chance of being true literally (interpreted as points), or less than 5% total chance of being true (interpreted as areas) I think the converse statement would have been more strictly true statistically (e.g. that 0.45 is statistically consistent with the data), and more accurately descriptive of what a p-test actually means.

LennyHenrysNuts 4y ago

So would this study have been able to show a massive uptick in myocarditis/pericarditis (if it had occurred)? Or is it pretty much useless? From my limited understanding, 2.56x next to none is still pretty close to next to none.

capnrefsmmat 4y ago
It would presumably be able to detect a massive increase, yes, if by "massive" you mean something like a 10x increase in rate. One could do a power calculation to determine its ability to detect different sizes of effect. Normally you'd do that when planning a study to determine the sample you need, but here they took every available patient in the medical records database they had access to, so they didn't really have a choice of sample size.

amluto 4y ago

I beg to differ. I think you and the paper are committing another falacy: getting hung up on relative instead of absolute hazards. Myocarditis and pericarditis (at least the reportable kind studied in the paper) are rare — all groups had incidence well under 0.01%. That’s 10 per 100k for those who prefer those units. The study enrolled almost 200k COVID patients and over 500k controls. This is a lot of power to bound the absolute risk, but it is indeed underpowered to get good bounds on the hazard ratio. The paper would be substantially improved if it said “the absolute risk of myo/pericarditis in COVID patients is <10 per 100k with confidence 99.9%” or “the increased risk is <9 per 100k with confidence such-and-such” (I made up the numbers, but the analysis is easy.) So yes, a 2x increased risk would be consistent with the data, but an actual substantial risk would not be consistent with the data.

capnrefsmmat 4y ago
Yes, I agree that it's important to put the absolute risk in context. The NEJM paper that started the concern about myocarditis put it this way: > Vaccination was most strongly associated with an elevated risk of myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9025013/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9025013/ Giving the risk difference in terms of events per 100,000 people is useful. It's also trickier to interpret because the length of the follow-up period is important: if the vaccine changes the rate of events, then a study that observes patients for two years after vaccination will have (roughly) double the number of events of a study that follows patients for one year after vaccination. That's why people like relative risks in this setting.