3 ms·
Interesting. This is something that I hoped someone would do. However, I think that for Benford's law to be meaningful, you probably need to be looking at the
by elihu 5y ago
Interesting. This is something that I hoped someone would do. However, I think that for Benford's law to be meaningful, you probably need to be looking at the finest-grain stats that are available, not just the top-line national numbers.
For instance, to give the U.S. as an example, the national numbers are a sum of state numbers (plus territories), which are themselves in most cases probably a sum of numbers from counties, which are perhaps sums from individual hospitals.
For Benford's law to work well I think you have to be looking at the same level where the fake numbers were created. If you add up a whole lot of fake numbers or a combination of real and fake numbers, the frequency distribution of the most significant digit of the total will probably end up being about the same as real data. For example, if fake numbers were being generated at the state level in four states, and at the county level in 30 different counties, and at 20 hospitals, then the national numbers would probably still look fine.
If we don't know whether fake data might be injected at the national, state, or county level or from individual hospitals then we'd have to check all of them. And if the data at the granularity we need to be looking at just isn't available then we probably won't be able to detect it just by applying Benford's law.
- LorenPechtel 5y agoBut as you drill down in detail you run out of enough cases to measure properly. Benford's law requires data that is distributed over a wide range--it's going to find problems with any data that is too granular. Hospital sizes aren't distributed over wide enough a range to apply Benford's law to. This came up not too long ago in looking for voting irregularities--somebody who didn't understand what they were doing applied Benford's law to precinct-level voting data--and of course it failed horribly. Duh! Voting precincts have an optimum size and thus will be pretty clustered around that size. The percent going to each candidate generally won't vary too widely, either--thus giving a substantial clustering of the numbers voting for each candidate. There's nowhere near the range of data to apply Benford's law to.