4 ms·

Before people start upvoting this because they think conspiracies are at play and the title is salacious, read the abstract: they're proposing a statistical tes

by timr 5y ago

Before people start upvoting this because they think conspiracies are at play and the title is salacious, read the abstract: they're proposing a statistical test and validating against countries with known undercounting. This is a methods paper: > We suggest a statistical test for underdispersion in the reported Covid-19 case and death numbers, compared to the variance expected under the Poisson distribution. Screening all countries in the World Health Organization (WHO) dataset for evidence of underdispersion yields 21 country with statistically significant underdispersion. Most of the countries in this list are known, based on the excess mortality data, to strongly undercount Covid deaths. We argue that Poisson underdispersion provides a simple and useful test to detect reporting anomalies and highlight unreliable data. A lot of countries aren't counting Covid deaths because they're poor and don't have good health systems. The title should really be changed to something less click-baitey, like "Statistical method for detecting undercounting of Covid-19 deaths"

darkcha0s 5y ago
From the Author: >I cannot think of any honest data collection issue that would result in such underdispersion. I believe this does automatically mean some amount of data tampering (which may not necessarily have malicious intent though; but for most countries on the list I suspect it does).
BitwiseFool 5y ago
The metrics and data collection around a subject so intensely tied to politics as Covid case/death numbers is bound to experience some measure of data manipulation. Especially when a large amount of funding and national prestige is at-play. To me, this is would be one of those perfectly believable conspiracy theories.
nebukadnet 5y ago
No, the paper identifies more countries than that. That's what makes the article interesting. > Most of the countries in this list are known, ...
- timr 5y ago
  It literally does not, except in the sense that it ran the method on a list of countries. From section 2.3: > Overall, 8 out of 10 countries with the highest undercount ratios in the World Mortality Dataset at the moment of writing demonstrated statistically significant underdispersion in our analysis. Translation: they ran it against a list of countries with known "undercount ratios", and found that 8/10 had high scores. Also: > The correlation of underdispersion index to undercount ratio was 0.40 (Figure 3). This is a poor overall correlation, and they do no sensitivity / specificity analysis so you can't jump to the conclusion that a score is meaningful. They make the argument that it works better for the countries with the highest undercounts...but if you know the countries have a high undercount you already know the answer by other methods. And even there, 80% sensitivity isn't that great.
logicalmonster 5y ago
> Most of the countries in this list are known, based on the excess mortality data, to strongly undercount Covid deaths. I take great issue with the word *known* up above. Somebody would need to do a deep dive into this data, but I'd say that this study is alarmingly starting with a premise that is not proven and may be completely false. If there are excess deaths in poor countries, are they likelier to come from Covid or economic devastation imposed by things like lockdowns, cutting off tourism income, and other economic restrictions? In rich countries, the average person was not close to not being able to acquire a meal at the start of Covid. In poor countries, even in good times being able to put food on the table is a real concern for average people, let alone after 2 years of super limited tourism and various lockdowns and economic restrictions that worsened many peoples' already precarious situations.
- timr 5y ago
  > I take great issue with the word known up above. It's a quote from the abstract. I didn't write it. > Somebody would need to do a deep dive into this data, but I'd say that this study is alarmingly starting with a premise that is not proven and may be completely false. This paper introduces no new data. It takes an existing data set, and runs a statistical analysis on it.
- whimsicalism 5y ago
  This is not really discussed (for somewhat obvious reasons), but economic decline is associated with a decline in all-cause mortality, not an increase.
derbOac 5y ago
I guess I still think it's interesting as a methods paper? It's also useful to have different sources of evidence about COVID underreporting. I agree that it seems to assume too much malice as a reason for underreporting in general though.
- timr 5y ago
  It's sort of interesting (it actually doesn't seem to be very good, with an R2 of 0.4 and 80% sensitivity at the high end), but I can pretty much guarantee from the comments here that people have no idea what it actually says. Just look at the sibling comments here -- most are about whether or not countries are under-reporting deaths. This is just more fuel for the eternal debate about Covid being bad.
  - asdfasgasdgasdg 5y ago
    As discussed in the paper: there are multiple known causes for undercounting, and this analysis, by design, only catches one of them. I.e. the two main causes for undercounting would be insufficient counting resources and deliberate manipulation, and this method is designed to detect only deliberate manipulation. For such an analysis, you would expect neither extremely high correlation nor extremely high sensitivity.
  - derbOac 5y ago
    I agree it's not the best paper, and there's lots of unrealistic assumptions being made, but I still liked to read it. If nothing else it's useful to see how these models map or don't map onto other sources of information. I also agree though that it would be nice if these sorts of things were approached differently, more from the perspective of epidemiological modeling or something.
asdfasgasdgasdg 5y ago
The title is fine as it is. It's not click baity at all, and it accurately reflects the study authors's views. I mean it is exactly the title of the study after all. "We believe that the most likely explanation for observed underdispersion patterns is deliberate data tampering,"

by timr 5y ago

darkcha0s 5y ago

From the Author: >I cannot think of any honest data collection issue that would result in such underdispersion. I believe this does automatically mean some amount of data tampering (which may not necessarily have malicious intent though; but for most countries on the list I suspect it does).

BitwiseFool 5y ago

The metrics and data collection around a subject so intensely tied to politics as Covid case/death numbers is bound to experience some measure of data manipulation. Especially when a large amount of funding and national prestige is at-play. To me, this is would be one of those perfectly believable conspiracy theories.

nebukadnet 5y ago

No, the paper identifies more countries than that. That's what makes the article interesting. > Most of the countries in this list are known, ...

timr 5y ago
It literally does not, except in the sense that it ran the method on a list of countries. From section 2.3: > Overall, 8 out of 10 countries with the highest undercount ratios in the World Mortality Dataset at the moment of writing demonstrated statistically significant underdispersion in our analysis. Translation: they ran it against a list of countries with known "undercount ratios", and found that 8/10 had high scores. Also: > The correlation of underdispersion index to undercount ratio was 0.40 (Figure 3). This is a poor overall correlation, and they do no sensitivity / specificity analysis so you can't jump to the conclusion that a score is meaningful. They make the argument that it works better for the countries with the highest undercounts...but if you know the countries have a high undercount you already know the answer by other methods. And even there, 80% sensitivity isn't that great.

logicalmonster 5y ago

> Most of the countries in this list are known, based on the excess mortality data, to strongly undercount Covid deaths. I take great issue with the word *known* up above. Somebody would need to do a deep dive into this data, but I'd say that this study is alarmingly starting with a premise that is not proven and may be completely false. If there are excess deaths in poor countries, are they likelier to come from Covid or economic devastation imposed by things like lockdowns, cutting off tourism income, and other economic restrictions? In rich countries, the average person was not close to not being able to acquire a meal at the start of Covid. In poor countries, even in good times being able to put food on the table is a real concern for average people, let alone after 2 years of super limited tourism and various lockdowns and economic restrictions that worsened many peoples' already precarious situations.

timr 5y ago
> I take great issue with the word known up above. It's a quote from the abstract. I didn't write it. > Somebody would need to do a deep dive into this data, but I'd say that this study is alarmingly starting with a premise that is not proven and may be completely false. This paper introduces no new data. It takes an existing data set, and runs a statistical analysis on it.
whimsicalism 5y ago
This is not really discussed (for somewhat obvious reasons), but economic decline is associated with a decline in all-cause mortality, not an increase.

derbOac 5y ago

I guess I still think it's interesting as a methods paper? It's also useful to have different sources of evidence about COVID underreporting. I agree that it seems to assume too much malice as a reason for underreporting in general though.

timr 5y ago
It's sort of interesting (it actually doesn't seem to be very good, with an R2 of 0.4 and 80% sensitivity at the high end), but I can pretty much guarantee from the comments here that people have no idea what it actually says. Just look at the sibling comments here -- most are about whether or not countries are under-reporting deaths. This is just more fuel for the eternal debate about Covid being bad.
- asdfasgasdgasdg 5y ago
  As discussed in the paper: there are multiple known causes for undercounting, and this analysis, by design, only catches one of them. I.e. the two main causes for undercounting would be insufficient counting resources and deliberate manipulation, and this method is designed to detect only deliberate manipulation. For such an analysis, you would expect neither extremely high correlation nor extremely high sensitivity.
- derbOac 5y ago
  I agree it's not the best paper, and there's lots of unrealistic assumptions being made, but I still liked to read it. If nothing else it's useful to see how these models map or don't map onto other sources of information. I also agree though that it would be nice if these sorts of things were approached differently, more from the perspective of epidemiological modeling or something.

asdfasgasdgasdg 5y ago

The title is fine as it is. It's not click baity at all, and it accurately reflects the study authors's views. I mean it is exactly the title of the study after all. "We believe that the most likely explanation for observed underdispersion patterns is deliberate data tampering,"