3 ms·

We have to be extremely cautious when considering any of the death-related statistics for this particular situation. For example, we know that a dubious method

We have to be extremely cautious when considering any of the death-related statistics for this particular situation. For example, we know that a dubious methodology was used for counting deaths in Toronto (Canada's most populous city, and the fourth most populous city in North America). This was publicly admitted to by Toronto Public Health itself, as early as June 2020: "Individuals who have died with COVID-19, but not as a result of COVID-19 are included in the case counts for COVID-19 deaths in Toronto." https://twitter.com/TOPublicHealth/status/1275888390060285967 https://twitter.com/TOPublicHealth/status/127588839006028596... I can't see how that significant city's death stats could be considered reliable or usable for analytic purposes, after considering how the counting was done. The provincial and national stats would also be affected to some extent. This makes me quite doubtful about the stats coming out of other regions, too.

shagie 4y ago
The paper those numbers are based on is Assessing the age specificity of infection fatality rates for COVID-19: systematic review, meta-analysis, and public policy implications https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7721859/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7721859/ The meta-analysis is looking at that question that you are asking. It also has all the papers that it cites for analysis and an analysis of its own limitations. > The systematic review encompassed 113 studies, of which 27 studies (covering 34 geographical locations) satisfied the inclusion criteria and were included in the meta-analysis.
cthulha 4y ago
The question nobody else has been able to answer is "What is a satisfactory analysis for these purposes?" The closest I've seen to an answer is to remove accidental deaths (eg, hit by a car, firearms fatality, etc) but there's a huge range of arguable cases like stroke which is plausibly covid-affected. Second, you have the problem of comparing stats between locations that used different definitions, so it's harder to do post-hoc correlations. The decisions seems to have been avoiding false-negative mistakes by increasing false positives in the first wave of analysis: "Let's include everything under a simple rule for covid stats so that we at least have some kind of worst-case baseline modelling with similar datasets, and we can figure out afterwards which are real and which aren't" Anyway, just wanted to see if you actually have a better answer to the problem than the standard that was used.
graeme 4y ago
You have to look at both sides of the equation. There are excess deaths in every region coinciding with Covid spikes. It takes enormous mental gymnastics to say “some deaths were over counted so deaths as a whole must be over counted” and completely discount the probability of missing deaths too. The recent Paxlovid study supports this. It reduces deaths days 30-90. Deaths that would not be counted as Covid deaths. But they clearly are if paxlovid reduces them.
- JamisonM 4y ago
  Yeah, it is just wacky stuff for people to be "COVID-19 NBD" while the hospitals are full and the morgues can't keep up with refrigeration requirements... it is not worth engaging with. The exact count doesn't matter, the spikes demonstrate the truth of the issue.
stephen_g 4y ago
I think that's a pretty specious argument though. Other statistics are available (especially excess deaths) that can pretty clearly help show what was going on along with the COVID death numbers (i.e. a significant increase in the number of people dying through the pandemic who most likely wouldn't have died during that time otherwise, in proportions generally tracking more or less to the reported COVID deaths). It's not like the reported numbers of COVID deaths exist in a vacuum to be able to be so sceptical of them!
rallison 4y ago
> This was publicly admitted to by Toronto Public Health itself, as early as June 2020: > "Individuals who have died with COVID-19, but not as a result of COVID-19 are included in the case counts for COVID-19 deaths in Toronto." I don't know the particulars of Toronto Public Health, but this wasn't uncommon for immediate reporting vs death record reporting. Since, in many places, comprehensive death records can take a few weeks (or longer!) to go through the pipeline before ending up on finalized reports, it's useful to have more immediate death reporting during a pandemic. Such immediate reporting is necessarily going to be a little rougher around the edges, but it generally gets you pretty close to the real numbers. But, it's absolutely true that this immediate reporting isn't perfect. Still, it's very useful, and usually close enough for near term needs. Taking the Toronto example, the main tradeoffs are that a) you include some deaths where Covid wasn't actually a contributing factor and b) you miss deaths where the person was never tested for Covid (especially applicable to deaths that occurred at home). Often, these two somewhat balanced themselves out, but only in places with fairly high levels of testing. Then, in the medium to longer term, you can switch over to relying on more comprehensive death records. This helps to filter out non-causal scenarios (e.g. the "hit by bus, tested positive for covid, died" scenarios). It also helps add in cases that were initially missed (had symptoms of covid, never went to hospital, died at home, etc). Finally, to validate numbers, we can also look at excess deaths. This helps ensure that we're not wildly off base with reported numbers. Excess deaths isn't a perfect metric to compare to, but it's still a great benchmark. To wrap up this comment, let me respond to this: > We have to be extremely cautious when considering any of the death-related statistics for this particular situation. I fully agree, but.. I would suggest that your comment missed a lot of the accuracy refinement that happens in practice over the medium to longer term. Additionally, it's very much worth pointing out that, with additional analysis (from death records, from excess death analysis, etc), we've found that very few places were likely over-reporting Covid deaths, but that a significant number of places were under-reporting Covid deaths. So, yes, take shorter term Covid death stats with a grain of salt, but know that we've generally ended up reporting too low, not too high.

shagie 4y ago

The paper those numbers are based on is Assessing the age specificity of infection fatality rates for COVID-19: systematic review, meta-analysis, and public policy implications https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7721859/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7721859/ The meta-analysis is looking at that question that you are asking. It also has all the papers that it cites for analysis and an analysis of its own limitations. > The systematic review encompassed 113 studies, of which 27 studies (covering 34 geographical locations) satisfied the inclusion criteria and were included in the meta-analysis.

cthulha 4y ago

The question nobody else has been able to answer is "What is a satisfactory analysis for these purposes?" The closest I've seen to an answer is to remove accidental deaths (eg, hit by a car, firearms fatality, etc) but there's a huge range of arguable cases like stroke which is plausibly covid-affected. Second, you have the problem of comparing stats between locations that used different definitions, so it's harder to do post-hoc correlations. The decisions seems to have been avoiding false-negative mistakes by increasing false positives in the first wave of analysis: "Let's include everything under a simple rule for covid stats so that we at least have some kind of worst-case baseline modelling with similar datasets, and we can figure out afterwards which are real and which aren't" Anyway, just wanted to see if you actually have a better answer to the problem than the standard that was used.

graeme 4y ago

You have to look at both sides of the equation. There are excess deaths in every region coinciding with Covid spikes. It takes enormous mental gymnastics to say “some deaths were over counted so deaths as a whole must be over counted” and completely discount the probability of missing deaths too. The recent Paxlovid study supports this. It reduces deaths days 30-90. Deaths that would not be counted as Covid deaths. But they clearly are if paxlovid reduces them.

JamisonM 4y ago
Yeah, it is just wacky stuff for people to be "COVID-19 NBD" while the hospitals are full and the morgues can't keep up with refrigeration requirements... it is not worth engaging with. The exact count doesn't matter, the spikes demonstrate the truth of the issue.

stephen_g 4y ago

I think that's a pretty specious argument though. Other statistics are available (especially excess deaths) that can pretty clearly help show what was going on along with the COVID death numbers (i.e. a significant increase in the number of people dying through the pandemic who most likely wouldn't have died during that time otherwise, in proportions generally tracking more or less to the reported COVID deaths). It's not like the reported numbers of COVID deaths exist in a vacuum to be able to be so sceptical of them!

rallison 4y ago

> This was publicly admitted to by Toronto Public Health itself, as early as June 2020: > "Individuals who have died with COVID-19, but not as a result of COVID-19 are included in the case counts for COVID-19 deaths in Toronto." I don't know the particulars of Toronto Public Health, but this wasn't uncommon for immediate reporting vs death record reporting. Since, in many places, comprehensive death records can take a few weeks (or longer!) to go through the pipeline before ending up on finalized reports, it's useful to have more immediate death reporting during a pandemic. Such immediate reporting is necessarily going to be a little rougher around the edges, but it generally gets you pretty close to the real numbers. But, it's absolutely true that this immediate reporting isn't perfect. Still, it's very useful, and usually close enough for near term needs. Taking the Toronto example, the main tradeoffs are that a) you include some deaths where Covid wasn't actually a contributing factor and b) you miss deaths where the person was never tested for Covid (especially applicable to deaths that occurred at home). Often, these two somewhat balanced themselves out, but only in places with fairly high levels of testing. Then, in the medium to longer term, you can switch over to relying on more comprehensive death records. This helps to filter out non-causal scenarios (e.g. the "hit by bus, tested positive for covid, died" scenarios). It also helps add in cases that were initially missed (had symptoms of covid, never went to hospital, died at home, etc). Finally, to validate numbers, we can also look at excess deaths. This helps ensure that we're not wildly off base with reported numbers. Excess deaths isn't a perfect metric to compare to, but it's still a great benchmark. To wrap up this comment, let me respond to this: > We have to be extremely cautious when considering any of the death-related statistics for this particular situation. I fully agree, but.. I would suggest that your comment missed a lot of the accuracy refinement that happens in practice over the medium to longer term. Additionally, it's very much worth pointing out that, with additional analysis (from death records, from excess death analysis, etc), we've found that very few places were likely over-reporting Covid deaths, but that a significant number of places were under-reporting Covid deaths. So, yes, take shorter term Covid death stats with a grain of salt, but know that we've generally ended up reporting too low, not too high.