5 ms·
I used Benford’s law to analyze Covid-19 in 113 countries (2020)
- deleted 5y ago[deleted]
- mulvya 5y ago> While today’s cumulative deaths is highly dependent on yesterday’s (it should theoretically never be less, for example), today’s new deaths isn’t as directly correlated to yesterday’s. Sort of. Assuming a wave pattern, values will be close to neighbouring values and vary in an gradual manner.
- not2b 5y agoExcept that the timing for reporting cases and deaths often varies considerably on weekends, which is why it is better to look at 7 day moving averages to smooth out that noise.
- deleted 5y ago[deleted]
- hammock 5y agoConcise writeup and useful case of Benford's law. Thanks to OP
- causi 5y agoWhat I found interesting is that his analysis did not pick up the (now strongly evidenced) degree of inaccurate reporting from India. Indications are that India's covid deaths are between six and seven times higher than reported yet it looks pretty good on the Benford chart.
- r00tanon 5y agoI'd you divided all your measurements by 7, I'm pretty sure both datasets would still align with Benford's curve.
- cogman10 5y agoFrom my coworkers in india, I think that India's reporting has been less to do with inaccuracy and more to do with their entire healthcare system just crumbling under the pressure of covid. Far too many people getting (and dying from) covid to accurately measure and capture the true impact. It was real bad.
- not2b 5y agoYes, lots of people died in rural areas and were buried without ever getting near a doctor in the Delta wave, so they weren't recorded.
- ceejayoz 5y agoNot saying India did this, but I would imagine someone aware of Benford's law could tweak numbers to look valid with it taken into account, right? The more folks talk about "I used Benford's law to spot malfeasance", the less useful it becomes.
- PennRobotics 5y agoIt depends how much the tweak implementer knows about Benford's law. If you look through the figures in the Appendix of https://econwpa.ub.uni-muenchen.de/econ-wp/othr/papers/0507/0507001.pdf https://econwpa.ub.uni-muenchen.de/econ-wp/othr/papers/0507/... , a lot of people get the first and sometimes second digit behavior right yet fail to properly falsify the less significant digits. Of course, this sort of analysis might fall apart once you've summed 20 or 40 or 300 days of data.
- bena 5y agoIt really depends on how you're manipulating data. If you're just making up numbers, Benford's law could pick it up (unless you're aware of Benford's law and create numbers to follow it). However, if you're just counting a sixth of all deaths, then your numbers will follow whatever pattern they were following before, just lower across the board. The general form of Benford's law also fails in situations where numbers are capped for some reason. You'll wind up with a strong grouping around that particular number. All in all, Benford's law can never prove anything. At most, it can give a good reason to look into a situation.
- causi 5y agoThis is an excellent point.
- ajross 5y agoBenford's law is sort of like the Bechdel test. It's trivial to evade if you know what algorithm for which to optimize; it's value is in demonstrating the surprising number of places where it fails.
- rossdavidh 5y agoThe CDC estimates that in the U.S., there are about 32 cases of undetected covid-19 for every 10 that are detected. That ratio (derived from testing of pregnant women, car crash victims, blood donors, etc.) was from before Omicron, when there was for a while a shortage of testing capacity. So, if the ratio in India is 60 to 10, while that is worse than in the US, it's not that much worse. In any case, Benford's Law helps to detect fraud (made-up numbers), not necessarily just inaccuracy (a low detection percentage, for example).
- deleted 5y ago[deleted]
- sudosysgen 5y agoThey're talking about deaths, not cases. For a case to go unreported is one thing, for a death not to be is much worse.
- tootie 5y agoBenford would likely detect manipulation, but not systemic underreporting. If they're just missing half the cases and accuracy is essentially arbitrary, then it probably wouldn't show up as violating Benford.
- r00tanon 5y agoI'd you divided all your measurements by 7, I'm pretty sure both datasets would still align with Benefords curve.
- belter 5y agoIn several Countries, the tax office will use Benford's law to decide if you are "randomly selected" for an audit. And it's quite useful to detect scientific fraud, and has been used that way several times. This is an interesting write-up, but would seem the diversity in both severity and duration of lock-downs, plus simple issues with quality of data collection mean here Benford's law anomalies, just help identify the lack of consistent approach between countries in handling the pandemic. Also the quality of their data collection process. This more than fraud, and in any case the article also makes that case.
- amelius 5y agoHere's another law: if you multiply two random integers, then the most likely outcome for the last digit is "0". The most likely outcome for the first digit is "1", but by a smaller margin.
- function_seven 5y agoHOW? I just did a toy trial with two random integers between 0 and 1,000,000. In Excel. I'm getting between 24% and 31% of the products ending in a '0'. Benford's law makes intuitive sense to me, in that as soon as you run through the leading digits 1-9, you then spend just as long as all those combined with a '1' in front of the "next set". But I always figured even digits would be more probable, but uniform across the [0,2,4,6,8] set.
- lucasmullens 5y agoWell, if either number ends in a 0, the product will end in a 0, so that's almost 20% of the cases right there. And if you get any even number and a number ending in 5 you'll get a product ending with 0 also, so 24-31% total sounds about right.
- function_seven 5y agoSuch a simple answer. Thanks. I just did a 10x10 times table and got the following prevalences: [27, 4, 12, 4, 12, 9, 12, 4, 12, 4] So '0' has 27% probability, '5' has 9%, and the rest of the evens 12%, the rest of the odds 4%
- inimino 5y agoAnother way to explain it is consider the prime factorization of both numbers. The product will contain all of the prime factors from both, so if there is at least one 2 and at least one 5 in then the result will be divisible by 10.
- brilee 5y agoif you draw out the 10x10 multiplication table, anything 0 * x or x * 0 ends in 0 (19 out of 100 cases), and any even number times 5 also ends in 0 (8 out of 100 cases), which lines up with the 27% you're seeing. Since this is on the units digit, Benford's law doesn't apply and you can basically expect the distribution of units digits to be uniform.
- dbodin11 5y agoTLDR - Article Highlights https://www.kontxt.io/document/d/nV9ATOU9PbvlwnbPlzXtdkouxX-FlFlWSHrczdlaB-KoS/summary?kontxt_user=d https://www.kontxt.io/document/d/nV9ATOU9PbvlwnbPlzXtdkouxX-...
- dang 5y agoWould you please stop doing this? It's repetitive and therefore off topic here. It's fine to occasionally post links to your own work when it's relevant. But it's not fine to do it routinely or generically—HN readers consider that spamming.
- dbodin11 5y agoSorry. I've checked HN several times a day every day for a decade and personally desired to see highlights of articles as a preview or summary since there's so much quality content here and so little time to look at it all. So I built Kontxt.io to save, organize, and share the best parts of the web. I thought others might be be interested and find it useful as well.
- dang 5y agoI totally get it, but posting generic links in the threads isn't the way to share the project. The more you do that, the more users (and eventually the software) are going to interpret it as spamming. I know it can be frustratingly difficult to get attention on HN, but if you're going to post a link in a thread, it needs to be in connection with something specifically relevant. Since you haven't posted a Show HN in two years, you should try again. "I built Kontxt.io to save, organize, and share the best parts of the web" is a good line - you should rework it (shortened) into your title. See https://news.ycombinator.com/item?id=22336638 https://news.ycombinator.com/item?id=22336638 for tips and https://news.ycombinator.com/yli.html https://news.ycombinator.com/yli.html for advice about how to present your work to HN. Email hn@ycombinator.com with a link when it's up, in case we can help. Good luck!
- sigmar 5y agoThis analysis doesn't make sense to me. China and other countries have had periods of double digit new cases reported. Even if that data is fake, you can't use Benford's law on numbers that small (it has to be numbers that span magnitudes... clearly states so in the wiki article he links). Also, cumulative stats aren't "naturally occurring random numbers."
- LorenPechtel 5y agoIn a country that hasn't done very much about the virus the cumulative stats will be distributed well enough. Note how all of the countries in the lower right part of the graph are the ones that managed to control it and thus pretty much freeze the cumulative number.
- smaryjerry 5y agoBenford’s law is something that’s not the most obvious why but I’ve tested it myself on even small sets of numbers like 1 years worth of transactions at a small company and it comes out really really close to shaping that curve. I’m was kind of surprised because it’s supposed to work on large datasets but I’m talking literally under a few hundred transactions where it’s often the same vendors repeating and the curve was still pretty close. So looking at this article I do believe something odd is likely going on where indicated.
- throwaway4good 5y agoUsing Benford's law on cumulative numbers is a bit silly, misleading even as you are sort of implying that some of fraud is going on. Suppose 5 people died of covid in the Vactican state before they were able to get it under control. The cummulative series is going to look something like this: 0 0 0 0 0 0 1 2 2 2 2 3 3 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 ... 5 being way overrepresented - without there being anything unnatural in it.
- generalizations 5y agoThis is explicitly discussed in the article. Why are you calling it misleading, when they make the same point as you do, in almost the same way? > For example, if like China, you eventually eradicate the virus when you have about 80,000 cases, you may end up with hundreds of days with values starting with 8. The article concludes by pointing out that for this reason, cumulative numbers aren't as suspicious, but anomalous daily numbers are...curious.
- throwaway4good 5y agoYou are right - it is actually mentioned in the article - sorry about that. I would still say that it is somewhat silly to apply Benford's law here - also in the non-cumulative case. The distribution of the daily number how much disease is circulating - where I am at the moment we have daily numbers stable at some 40-50 thousands, leading to 4 and 5 being overrepresented, without it being in any way being fraudulent or made up.
- BeetleB 5y agoAny time a random person invoked Benford's Law, be wary. It's often misapplied for fraud detection, and it's use in any new arena has to be justified. You can't simply take a bunch of data and assume it should follow Benford's Law.
- elihu 5y agoInteresting. This is something that I hoped someone would do. However, I think that for Benford's law to be meaningful, you probably need to be looking at the finest-grain stats that are available, not just the top-line national numbers. For instance, to give the U.S. as an example, the national numbers are a sum of state numbers (plus territories), which are themselves in most cases probably a sum of numbers from counties, which are perhaps sums from individual hospitals. For Benford's law to work well I think you have to be looking at the same level where the fake numbers were created. If you add up a whole lot of fake numbers or a combination of real and fake numbers, the frequency distribution of the most significant digit of the total will probably end up being about the same as real data. For example, if fake numbers were being generated at the state level in four states, and at the county level in 30 different counties, and at 20 hospitals, then the national numbers would probably still look fine. If we don't know whether fake data might be injected at the national, state, or county level or from individual hospitals then we'd have to check all of them. And if the data at the granularity we need to be looking at just isn't available then we probably won't be able to detect it just by applying Benford's law.
- LorenPechtel 5y agoBut as you drill down in detail you run out of enough cases to measure properly. Benford's law requires data that is distributed over a wide range--it's going to find problems with any data that is too granular. Hospital sizes aren't distributed over wide enough a range to apply Benford's law to. This came up not too long ago in looking for voting irregularities--somebody who didn't understand what they were doing applied Benford's law to precinct-level voting data--and of course it failed horribly. Duh! Voting precincts have an optimum size and thus will be pretty clustered around that size. The percent going to each candidate generally won't vary too widely, either--thus giving a substantial clustering of the numbers voting for each candidate. There's nowhere near the range of data to apply Benford's law to.