3 ms·

A good candidate could be Benford's law [1], which makes predictions about the distribution of digits. If the digits of the numbers found in the results section

A good candidate could be Benford's law [1], which makes predictions about the distribution of digits. If the digits of the numbers found in the results section of the paper deviate too much from this law, it could be a red flag. [1] https://en.wikipedia.org/wiki/Benford%27s_law https://en.wikipedia.org/wiki/Benford%27s_law

anonymous_sorry 4y ago
Benford's "law" doesn't apply to everything, so it would depend a lot on the field and phenomenon being studied. And to be honest, once it was known to be used in this way, it would be simple to make fake data comply with the law.
- bell-cot 4y ago
  True. But the "once it was known...make fake data to comply" issue applies to ~every fake-detecting method which could be applied. Big picture, the goal is not to catch every fake. It's to (1) catch enough so that smarter fakers learn to submit their stuff elsewhere (vs. to the journal, conference, or institution in question, which really cares about fake detection). And (2) dimmer fakers are mostly caught ~promptly.
duskwuff 4y ago
It looks more like the papers in question are taking a large public dataset and performing some statistical analysis on it, often along the lines of showing a correlation between metrics which represents pollution (like CO2 emissions) and metrics which represent economic growth. The source data is real, and the analysis is probably real too; neither is likely to trip Benford's law. The problem is that there's nothing novel being uncovered by these articles -- it will surprise no one, for example, to hear that China's CO2 emissions have risen in conjunction with their energy usage and economic growth (https://doi.org/10.1007/s11356-020-12217-6 https://doi.org/10.1007/s11356-020-12217-6). Substituting China for Pakistan, or energy usage for foreign remittances, doesn't make for novel research; it's simply plugging in another set of numbers and turning the crank.

anonymous_sorry 4y ago

Benford's "law" doesn't apply to everything, so it would depend a lot on the field and phenomenon being studied. And to be honest, once it was known to be used in this way, it would be simple to make fake data comply with the law.

bell-cot 4y ago
True. But the "once it was known...make fake data to comply" issue applies to ~every fake-detecting method which could be applied. Big picture, the goal is not to catch every fake. It's to (1) catch enough so that smarter fakers learn to submit their stuff elsewhere (vs. to the journal, conference, or institution in question, which really cares about fake detection). And (2) dimmer fakers are mostly caught ~promptly.

duskwuff 4y ago

It looks more like the papers in question are taking a large public dataset and performing some statistical analysis on it, often along the lines of showing a correlation between metrics which represents pollution (like CO2 emissions) and metrics which represent economic growth. The source data is real, and the analysis is probably real too; neither is likely to trip Benford's law. The problem is that there's nothing novel being uncovered by these articles -- it will surprise no one, for example, to hear that China's CO2 emissions have risen in conjunction with their energy usage and economic growth (https://doi.org/10.1007/s11356-020-12217-6 https://doi.org/10.1007/s11356-020-12217-6). Substituting China for Pakistan, or energy usage for foreign remittances, doesn't make for novel research; it's simply plugging in another set of numbers and turning the crank.