The PSU Grievance Studies Affair [1] had the same objective as the 1996 Sokal Affair: to test peer review by submitting obviously bad papers that should be reje

The PSU Grievance Studies Affair [1] had the same objective as the 1996 Sokal Affair: to test peer review by submitting obviously bad papers that should be rejected, and seeing how many of them are accepted. The motivation behind this is entirely reasonable. Bad "true negative" as well as known-good "true positive" articles ought to be regularly submitted to every peer review process as a test of accuracy. A great example of a similar idea was the 2014 NIPS consistency experiment [2]. However, there is a problem with the "dog rape" hoax article [3] as a test of the peer review system. Specifically, the author in that paper claims to have spent over 1000 hours carefully cataloguing the behavior of ten thousand people and dogs over the course of one year. The paper then proceeds to produce silly but surprising statistics: female dogs are 70% more likely to be leashed than male dogs, 100% of dogs with shock collars are male, 847 instances of dogs fighting were observed, and so on. The problem here is that the data was falsified: the observations and experiments claimed never happened, the numbers were all made up. The article was likely accepted because of its data, but the data was a lie. If this data had been real, it might have been a minor but useful study relevant to economists, sociologists, or urban planners, regardless of how silly or made-up the conclusions at the end were. Peer review works based on the assumption that the author is telling the truth about what experiments they conducted and what numbers they measured. The grievance studies authors could have submitted fake and silly hoax papers without falsifying data: that's what Sokal did, and that would have been a valid experiment testing the peer review system. But that is not what the authors chose to do. [1] https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair [2] http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html [3] http://norskk.is/bytta/menn/dog_park.pdf http://norskk.is/bytta/menn/dog_park.pdf