4 ms·

A great method for spotting (or at least suspecting) fake data is to see if it follows Benford's law. (So remember to fit your fake data to conform!) http://en

A great method for spotting (or at least suspecting) fake data is to see if it follows Benford's law. (So remember to fit your fake data to conform!) http://en.wikipedia.org/wiki/Benford's_law http://en.wikipedia.org/wiki/Benford's_law > Benford's Law, also called the First-Digit Law, refers to the frequency distribution of digits in many (but not all) real-life sources of data. In this distribution, the number 1 occurs as the leading digit about 30% of the time, while larger numbers occur in that position less frequently: 9 as the first digit less than 5% of the time. Benford's Law also concerns the expected distribution for digits beyond the first, which approach a uniform distribution. As one might imagine: > A test of regression coefficients in published papers showed agreement with Benford's law. As a comparison group subjects were asked to fabricate statistical estimates. The fabricated results failed to obey Benford's law. So keep that in mind, data fabricators!

DangerousPie 13y ago
That only helps you spot outright fabrication though, doesn't it? I would assume that this only happens in the minority of cases. The much bigger issue is people trying many different statistical tests until they find the one that "works", removing outliers to make the data look better or just neglecting to mention things like p-values or confidence intervals in the hope that nobody would notice. Much harder to spot, but potentially as misleading as just making numbers up...
- Zoomla 13y ago
  most real datasets apparently have outliers... you should not remove them if you want it to look real (I heard this from this course: https://class.coursera.org/dataanalysis-002/ https://class.coursera.org/dataanalysis-002/)
Zoomla 13y ago
(but probably don't fit it exactly)
astine 13y ago
You have to be careful with this though. Benford's law only holds for certain types of data, such as growing populations. It has to do with the nature of the decimal system.
- yen223 13y ago
  It has more to do with the fact that a good chunk of natural phenomenon are distributed exponentially, not linearly. Benford's Law actually applies to other notational systems.
- Terr_ 13y ago
  To be more specific, it has to do with the nature of any positional-notation system, whether it is decimal, octal, hex, or even binary.
  - T-hawk 13y ago
    Wow, thinking about Benford's Law in binary is really interesting, thanks for this comment. It's the most extreme expression of the concept. Every number in binary starts with 1. The only other possible leading digit would be a meaningless leading zero, so every number starts with 1. Benford's Law holds in the most extreme manner possible in binary notation, that numbers starting with 1 dominate the population over all others.
Terr_ 13y ago
Just to explain this a bit, it has to do with relative growth/shrinkage and the base of the positional-numbering system you're using. If you have a random starting value (X) multiplied by a second random factor (Y), most of the time the result will start with a one. You're basically throwing darts at logarithmic graph paper! The area covered by squares which "start with 1" is larger than the area covered by square which "start with 9".
- function_seven 13y ago
  > You're basically throwing darts at logarithmic graph paper! The area covered by squares which "start with 1" is larger than the area covered by square which "start with 9". Best explanation for Benford's Law I've ever seen. I've always had a rough understanding of the "why" of it, but this really drives the point home.

DangerousPie 13y ago

That only helps you spot outright fabrication though, doesn't it? I would assume that this only happens in the minority of cases. The much bigger issue is people trying many different statistical tests until they find the one that "works", removing outliers to make the data look better or just neglecting to mention things like p-values or confidence intervals in the hope that nobody would notice. Much harder to spot, but potentially as misleading as just making numbers up...

Zoomla 13y ago
most real datasets apparently have outliers... you should not remove them if you want it to look real (I heard this from this course: https://class.coursera.org/dataanalysis-002/ https://class.coursera.org/dataanalysis-002/)

Zoomla 13y ago

(but probably don't fit it exactly)

astine 13y ago

You have to be careful with this though. Benford's law only holds for certain types of data, such as growing populations. It has to do with the nature of the decimal system.

yen223 13y ago
It has more to do with the fact that a good chunk of natural phenomenon are distributed exponentially, not linearly. Benford's Law actually applies to other notational systems.
Terr_ 13y ago
To be more specific, it has to do with the nature of any positional-notation system, whether it is decimal, octal, hex, or even binary.
- T-hawk 13y ago
  Wow, thinking about Benford's Law in binary is really interesting, thanks for this comment. It's the most extreme expression of the concept. Every number in binary starts with 1. The only other possible leading digit would be a meaningless leading zero, so every number starts with 1. Benford's Law holds in the most extreme manner possible in binary notation, that numbers starting with 1 dominate the population over all others.

Terr_ 13y ago

Just to explain this a bit, it has to do with relative growth/shrinkage and the base of the positional-numbering system you're using. If you have a random starting value (X) multiplied by a second random factor (Y), most of the time the result will start with a one. You're basically throwing darts at logarithmic graph paper! The area covered by squares which "start with 1" is larger than the area covered by square which "start with 9".

function_seven 13y ago
> You're basically throwing darts at logarithmic graph paper! The area covered by squares which "start with 1" is larger than the area covered by square which "start with 9". Best explanation for Benford's Law I've ever seen. I've always had a rough understanding of the "why" of it, but this really drives the point home.