11 ms·

This was what a few academics hypothesized and set out to prove in what is now known as the Grievance Studies Affair [1]. Not only did they get multiple hoax pa

This was what a few academics hypothesized and set out to prove in what is now known as the Grievance Studies Affair [1]. Not only did they get multiple hoax papers published, they got recognized for “excellent scholarship” [2]. [1] https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair [2] https://youtu.be/kVk9a5Jcd1k?t=3m49s https://youtu.be/kVk9a5Jcd1k?t=3m49s

fenomas 3y ago
That grievance studies thing was idiotic, and really ought to be dismissed and forgotten. Reasons: 1. The hoax authors didn't submit gibberish or anything, like with Sokal - they submitted coherent papers that they considered absurd, but which they expected the journals to find reasonable. As such, they only thing they can even pretend to have proven is that they and the journals disagree about what's reasonable. 2. IIRC, every one of the initial hoax papers was flat-out rejected! If the authors had any backbone they could have stopped right there, and published a paper called "These grievance journals aren't as bad as we expected." :D But they didn't - they set out to iterate on each paper until a journal accepted it. And if your experimental methodology is "keep trying different things until X happens", you can hardly pretend to have proved something when X happens. I mean, I agree with the premise the hoax authors were trying to prove. But their "experiment" was silly, and I think it's dishonest of them to pretend it proved something. Meh. Meh!
- foldr 3y ago
  Excellent summary. I’d also mention the lack of a control group. We’re supposed to conclude that it’s easier to (eventually) get fraudulent results published in so-called ‘grievance’ fields than it is in (say) physics. But the hoax authors never made any effort to publish fraudulent results in those fields, so we can’t know if the percentage of papers that were eventually accepted would have been lower or not.
  - jballer 3y ago
    We were typing at the same time - see my other comment (sibling to yours) linking to Thiel arguing things are likely worse in hard sciences, and encouraging people to find ways to expose it.
- jballer 3y ago
  Even so, it was a good demonstration of the point above: the peer review process in journals is not designed to keep out garbage, and is incentivized to apply less scrutiny when the subject is politically convenient. Peter Thiel in this recent talk argues by inference that this problem is likely deeper and more pervasive in hard sciences than in the humanities. https://youtu.be/lO0KH-TgvbM?t=22m20s https://youtu.be/lO0KH-TgvbM?t=22m20s
  - foldr 3y ago
    Any evaluation metric is subject to political influences. Whatever the exact mechanism, it's people who are evaluating the papers, and people have political views. The time Thiel pines for is simply the time when the US government spent more money funding science, so that there was less competition to get funding. https://ssti.org/blog/changing-nature-us-basic-research-trends-federal-spending https://ssti.org/blog/changing-nature-us-basic-research-tren...
  - fenomas 3y ago
    That's my issue with it, really - I think it was widely presented and received as having "demonstrated" something, but if you look closely, by reasonable standards it didn't. Actually if you dig up the authors' initial article about it, IIRC at the end there's a line like "so what does any of this actually prove? Well, we'll leave that for you to decide." If the authors can't say what their experiment proved, I think it's reasonable to conclude it didn't prove anything :D
- tomsthumb 3y ago
  >they submitted coherent papers that they considered absurd https://en.wikipedia.org/wiki/Affilia https://en.wikipedia.org/wiki/Affilia
  - foldr 3y ago
    >It was later reported that the manuscript included plagiarized sections from Chapter 12 of Adolf Hitler's Mein Kampf (My Struggle), in which Hitler describes why the Nazi Party is needed and what it requires of its members. The authors replaced Hitler's references to "National Socialism" with "feminism" and "Jews" with "privilege". Isn't this a pretty good example of how crude the hoax was? Of course some things that Hitler said won't be awful if you literally change the key words in the sentences. Imagine if I played this trick the other way round. I take something innocuous that a politician has said and replace random words or phrases with 'National Socialism' and 'Jews'. Suddenly what they're saying seems really controversial! And this shows...what?
    - tomsthumb 3y ago
      >they submitted coherent papers that they considered absurd I’ve never heard anyone describe Mein Kampf as coherent. The ramblings of an asshole clearly abusing stimulants would be closer to the usual. >Of course some things that Hitler said won't be awful if you literally change the key words in the sentence The idea that swapping around the group being discussed is all it would take to make it not awful is, well, novel if nothing else.
      - foldr 3y ago
        Do you happen to have the passage in question at hand? Mein Kampf is a fairly long book, and it seems likely to me that there are passages of it which – with key words changed! – would not express anything particularly awful. But it is hard to judge without knowing what the modified passage actually said. Edit: What the hoaxers actually say suggests that the passages were altered quite substantially: "The last two thirds of this paper is based upon a rewriting of roughly 3600 words of Chapter 12 of Volume 1 of Mein Kampf, by Adolf Hitler, though it diverges significantly from the original.". And you can read the paper here: https://newdiscourses.com/feminist-mein-kampf/#solidarity https://newdiscourses.com/feminist-mein-kampf/#solidarity
Beldin 3y ago
"Prove". From what I have read so far, they showed that if you completely fabricate the data and follow the advice of peer reviewers, you can end up at a published paper. Thank you, Captain Obvious. Had these valiant warriors heard of Diederik Stapel, they would have known that fraudsters already made this point. Extensively. These fighters for academic rigour seem to have lost sight of their own academic rigour in this "experiment". Peer review is insufficient to catch fraudulent submissions. This is not a surprise as much as a misunderstanding of what peer review is. To be clear: to show something, the authors should have shown that the ideas were obviously false, not that fraudulent data that supports findings would go undetected. Einstein's famous works were considered patently absurd at the time. The only way to substantiate or refute such ideas is with data. An experiment that abuses this to "show" anything is, itself, patently absurd.
- jballer 3y ago
  Point taken, I could’ve worded my comment better

fenomas 3y ago

That grievance studies thing was idiotic, and really ought to be dismissed and forgotten. Reasons: 1. The hoax authors didn't submit gibberish or anything, like with Sokal - they submitted coherent papers that they considered absurd, but which they expected the journals to find reasonable. As such, they only thing they can even pretend to have proven is that they and the journals disagree about what's reasonable. 2. IIRC, every one of the initial hoax papers was flat-out rejected! If the authors had any backbone they could have stopped right there, and published a paper called "These grievance journals aren't as bad as we expected." :D But they didn't - they set out to iterate on each paper until a journal accepted it. And if your experimental methodology is "keep trying different things until X happens", you can hardly pretend to have proved something when X happens. I mean, I agree with the premise the hoax authors were trying to prove. But their "experiment" was silly, and I think it's dishonest of them to pretend it proved something. Meh. Meh!

foldr 3y ago
Excellent summary. I’d also mention the lack of a control group. We’re supposed to conclude that it’s easier to (eventually) get fraudulent results published in so-called ‘grievance’ fields than it is in (say) physics. But the hoax authors never made any effort to publish fraudulent results in those fields, so we can’t know if the percentage of papers that were eventually accepted would have been lower or not.
- jballer 3y ago
  We were typing at the same time - see my other comment (sibling to yours) linking to Thiel arguing things are likely worse in hard sciences, and encouraging people to find ways to expose it.
jballer 3y ago
Even so, it was a good demonstration of the point above: the peer review process in journals is not designed to keep out garbage, and is incentivized to apply less scrutiny when the subject is politically convenient. Peter Thiel in this recent talk argues by inference that this problem is likely deeper and more pervasive in hard sciences than in the humanities. https://youtu.be/lO0KH-TgvbM?t=22m20s https://youtu.be/lO0KH-TgvbM?t=22m20s
- foldr 3y ago
  Any evaluation metric is subject to political influences. Whatever the exact mechanism, it's people who are evaluating the papers, and people have political views. The time Thiel pines for is simply the time when the US government spent more money funding science, so that there was less competition to get funding. https://ssti.org/blog/changing-nature-us-basic-research-trends-federal-spending https://ssti.org/blog/changing-nature-us-basic-research-tren...
- fenomas 3y ago
  That's my issue with it, really - I think it was widely presented and received as having "demonstrated" something, but if you look closely, by reasonable standards it didn't. Actually if you dig up the authors' initial article about it, IIRC at the end there's a line like "so what does any of this actually prove? Well, we'll leave that for you to decide." If the authors can't say what their experiment proved, I think it's reasonable to conclude it didn't prove anything :D
tomsthumb 3y ago
>they submitted coherent papers that they considered absurd https://en.wikipedia.org/wiki/Affilia https://en.wikipedia.org/wiki/Affilia
- foldr 3y ago
  >It was later reported that the manuscript included plagiarized sections from Chapter 12 of Adolf Hitler's Mein Kampf (My Struggle), in which Hitler describes why the Nazi Party is needed and what it requires of its members. The authors replaced Hitler's references to "National Socialism" with "feminism" and "Jews" with "privilege". Isn't this a pretty good example of how crude the hoax was? Of course some things that Hitler said won't be awful if you literally change the key words in the sentences. Imagine if I played this trick the other way round. I take something innocuous that a politician has said and replace random words or phrases with 'National Socialism' and 'Jews'. Suddenly what they're saying seems really controversial! And this shows...what?
  - tomsthumb 3y ago
    >they submitted coherent papers that they considered absurd I’ve never heard anyone describe Mein Kampf as coherent. The ramblings of an asshole clearly abusing stimulants would be closer to the usual. >Of course some things that Hitler said won't be awful if you literally change the key words in the sentence The idea that swapping around the group being discussed is all it would take to make it not awful is, well, novel if nothing else.
    - foldr 3y ago
      Do you happen to have the passage in question at hand? Mein Kampf is a fairly long book, and it seems likely to me that there are passages of it which – with key words changed! – would not express anything particularly awful. But it is hard to judge without knowing what the modified passage actually said. Edit: What the hoaxers actually say suggests that the passages were altered quite substantially: "The last two thirds of this paper is based upon a rewriting of roughly 3600 words of Chapter 12 of Volume 1 of Mein Kampf, by Adolf Hitler, though it diverges significantly from the original.". And you can read the paper here: https://newdiscourses.com/feminist-mein-kampf/#solidarity https://newdiscourses.com/feminist-mein-kampf/#solidarity

Beldin 3y ago

"Prove". From what I have read so far, they showed that if you completely fabricate the data and follow the advice of peer reviewers, you can end up at a published paper. Thank you, Captain Obvious. Had these valiant warriors heard of Diederik Stapel, they would have known that fraudsters already made this point. Extensively. These fighters for academic rigour seem to have lost sight of their own academic rigour in this "experiment". Peer review is insufficient to catch fraudulent submissions. This is not a surprise as much as a misunderstanding of what peer review is. To be clear: to show something, the authors should have shown that the ideas were obviously false, not that fraudulent data that supports findings would go undetected. Einstein's famous works were considered patently absurd at the time. The only way to substantiate or refute such ideas is with data. An experiment that abuses this to "show" anything is, itself, patently absurd.

jballer 3y ago
Point taken, I could’ve worded my comment better