6 ms·

I think back to the recent group of academics that wanted to prove how most Grievance Studies journals don't property review submissions. One of the papers they

I think back to the recent group of academics that wanted to prove how most Grievance Studies journals don't property review submissions. One of the papers they got accepted had chapters from Mein Kampf rewritten with terminology from modern social cause rhetoric. Instead of accepting the issues, with the entire process, these professors are now facing disciplinary hearings for submitting false research. In a way I can understand this; they did after all submit papers they knew weren't academic (in bad faith?) in order to prove a greater point. But does that justify what they did?

michaelmrose 8y ago
Yes
- notduncansmith 8y ago
  I also believe what they did was justified. They did actual science and revealed that a number of publishers within this field demonstrate such a strong preference for virtue-signaling papers that they have lowered the bar for critical thought in both content and review. One of the honeypot papers won an award. That shows you how obvious the publisher bias is, and how obvious it must have been to these people before they decided to do something about it. Not only did these individuals risk their careers to help save a dying system with this project, they held themselves to a pretty high standard of execution. We place lots of limits on the amount of heroism individuals are allowed to display, and I think that’s mostly beneficial - but sometimes we need heroes.
gizmo686 8y ago
I'm not an academic, but aren't there standard practices for doing that sort of ethically questionable research. Namely, an ethics committee that you go to for the green light before proceeding.
- learc83 8y ago
  There's usually an institutional review board that you go through when using research subjects, but you don't need to do that to publish something like a new algorithm you developed in a CS journal. In this case, the researcher probably should have gotten IRB approval because there were test subjects--the journal editors, peer reviewers, and probably journal readers. But I'm assuming they just skipped that process and said there were no human subjects.
Baeocystin 8y ago
>In a way I can understand this; they did after all submit papers they knew weren't academic (in bad faith?) in order to prove a greater point. But does that justify what they did? A thousand times yes. Exposing cargo cults that attempt to co-opt the hard-earned cachet of genuine scientific inquiry for their own ideological ends is exactly what is needed to keep science functional.
- lapnitnelav 8y ago
  Yeah but it's a thin line to walk. If you don't do it, then you let all the pseudo-science out there masquerading as real science, which is bad. But if you overdo it, you feed into the distrust of proper science too, see Brexit and we're done listening to experts soundbite, or any of the anti-vax crowd.
  - Baeocystin 8y ago
    In my experience, people that have profoundly anti-scientific beliefs have always been such; whether their purported trigger is Jenny McCarthy or an odd alignment of Jupiter, it's hard to know if it's even relevant- the problem is that they don't have the skillset to make rational choices. And I just don't see a Sokal-style hoax making a whit of difference in their minds, as there are a million other things that could do the same. In other words, their feelings of anti-science came first. Whatever reason provided is ex post facto confabulation. Meanwhile, the sciences themselves need this kind of scrutiny to stay scientific. I'm open to being wrong. But I think it is a mistake to put too much weight to the 'we should be careful what the people who never use science anyway' consider.
shrewduser 8y ago
there are a number of fields that seem to operate on motivated reasoning over objective truth.
pasabagi 8y ago
These kind of stunts are pretty annoying, though. I know with the original (who was it again?), it was essentially a case of the academic leveraging his reputation, the editors being a bit like 'this seems kooky, but sure, you're the expert', then him turning around and pretending like he'd fooled everyone.
- hnuser123456 8y ago
  It was by James Lindsay, Peter Boghossian, and Helen Pluckrose. They submitted papers with fictitious names and institutions. They were not leveraging their own reputations (besides the publication's, if they were accepted, and a Richard Baldwin who lent his identity for some of them.) They spent time reviewing and citing existing work, so they could legitimately claim to be building on it, but then try to go in the most absurd directions possible, so absurd that any self-respecting academic publication ought to immediately reject it with disgust, and see what happened. 7 out of 20 such papers were accepted, 4 were published, they received 4 invitations to peer-review other papers, and 1 "gained special recognition for excellence." https://areomagazine.com/2018/10/02/academic-grievance-studies-and-the-corruption-of-scholarship/ https://areomagazine.com/2018/10/02/academic-grievance-studi...
  - hyeonwho4 8y ago
    ... and they were only discovered to be frauds by the twitter feeds that mock social sciences research.
  - pasabagi 8y ago
    If I published something in any of the innumerable scientific 'journals' that bombard my spam box, I could probably publish whatever I liked. That doesn't mean I can say science is methodologically flawed. The humanities, no doubt, deserve criticism, but I honestly don't see the appeal of stunts designed to 'prove' that a certain field is not worth the time of day, by people who don't even understand the field at the most basic level. How would they know? Why would they care? It strikes me as embarrassing for everybody involved- embarrassing, to publish something without properly vetting it, embarrassing, to dismiss something as nonsense without actually having the measure of it, embarrassing, to be so enthusiastic about this kind of 'proof', which so obviously combines ignorance with arrogance. There is no such thing as grievance studies. Critical theory has nothing to do with identity. The whole thing is about as close to the mark as Pat Buchanan's critique of feminism, and about as clever. In short, depressing.
pfortuny 8y ago
Of course it is justified: you can only prove that kind of abuse by making it glare: otherwise, the “secrecy” of the peer-review process is the excuse.
bartread 8y ago
> But does that justify what they did? Yes, it does. These journals are an absolute joke, make a mockery of any serious attempt to advance knowledge and understanding, and deserve to be pilloried. Their standards of rigour and review are shockingly poor and as such they do not provide a suitable vehicle for the publication of genuine research, or for measured consideration of the content they do publish. Just this week I ran across this, where a couple of the academics concerned were interviewed by Joe Rogan late last year: https://www.youtube.com/watch?v=AZZNvT1vaJg&t=1s https://www.youtube.com/watch?v=AZZNvT1vaJg&t=1s Fascinating, horrifying, and entertaining in equal measure. For more background, there's also this on wikipedia: https://en.wikipedia.org/wiki/Grievance_Studies_affair https://en.wikipedia.org/wiki/Grievance_Studies_affair
- pfortuny 8y ago
  Thank you for your clarity. We need more of this.
CJefferson 8y ago
It doesn't justify what they did, and what they did has very little value, for 4 reasons. 1) there are ethics panels for a reason -- many people have run unacceptable experiments on people. Academics don't get to skip the ethics checks because fhey feel like it. 2) their experiment was poorly designed. On their first attempt of their papers got ejected, so they polished and resubmitted them. What is their exact scientific theory? 3) they were clearly trying for contraversy. Maybe some part of Mein Kampf are actually useful valid statements to make. Just because of the terribleness of the Nazis, doesn't mean everything Hitler ever said, or wrote, should automatically be rejected from journals. 4) maybe Grievence Studies actually review better than everyone else. Any good study need a control.
- kristianc 8y ago
  > 2) their experiment was poorly designed. On their first attempt of their papers got ejected, so they polished and resubmitted them. What is their exact scientific theory? They've outlined this. There is a specific Grievance Studies nomenclature and orthodoxy which the first papers did not adhere to. When they republished and resubmitted, they followed the Orthodoxy and they got through. That IS the scientific method. You run an experiment, observe the results. You tweak a few variables and see what happens under those conditions. It’s what distinguishes their work from Grievance Scholarship, where you know the conclusion you want before you start. > Maybe some part of Mein Kampf are actually useful valid statements to make. Um?
  - CJefferson 8y ago
    Let's pick a random bit of Mein Kampf (I just scrolled to a random page in the middle in a randomly chosen translation). This looks, to me, like something that wouldn't be out of place in any discussion of "Fake News". This is the problem with using bits of Mein Kampf, it feels designed to shock and offend. If I got this text below into an article in a major newspaper, would that make that newspaper fundamentally broken / untrustworthy, because it happens to come, word for word, from Mein Kampf? Quote begins: Just in journalistic circles one usually prefers to call the press a 'great power' of the State. As a matter of fact its importance is truly enormous. It cannot be overestimated ; it is indeed actually the continuation of the education of youth in advanced age. Thereby one can divide the readers as a whole into three groups: First, those who believe everything they read; Secondly, those who no longer believe anything; Thirdly, those who critically examine what they have read and judge accordingly.
ATsch 8y ago
I think it's surprising that on a site such as HN the narrative of these authors is so readily accepted without any second thought. The claim is that they just made up trash and submitted it to high-quality peer-reviewed journals and they accepted them. The reality is much less outragous. Of the 20 papers, 4 were published and 3 more accepted for publication. They spent a year doing this. The authors note that they submitted the papers to "top journals in the field", leaving out the fact that all of those top journals rejected the 20 papers and they had to revise the papers and send them to shoddier journals instead. The review quoted by the authors was from an unsure grad student doing their first review of a paper. Said student later said they knew the paper was crap, but still tried to be constructive. The assertion of the authors is that this stuff getting published means the whole field is worthless. Which is of course ridiculous. Medicine didn't become obsolete when a journal published a paper with an abstract ending in "The fact that these last sentences appear in the published paper tell you, dear reader, exactly how seriously the editorial process has been taken". Computer Science didn't implode when three MIT students made an AI to generate random papers and 120(!) were accepted by the reputable IEEE and Springer. The blame was back then correctly put on the publishers and the general flaws of the current scientific process instread. So, as people have pointed out, it's ironic that their essay itself is incredibly scientifically unrigorous. No controls, unsourced claims, biased authors and conclusions that aren't supported by the data.
- prepend 8y ago
  “The assertion of the authors is that this stuff getting published means the whole field is worthless.” I don’t get this at all from the authors. My take is that they call out serious flaws in the rigor of how papers are reviewed. I don’t think it’s reasonable to extend that to say the field is worthless. I did read through many papers in the referenced journals and it seems really surprising to me. I’d love to see some of the research reproduced. But I’m not even sure how you could scientifically prove a field worthless.
  - ATsch 8y ago
    > I don’t get this at all from the authors. My take is that they call out serious flaws in the rigor of how papers are reviewed. I don’t think it’s reasonable to extend that to say the field is worthless. I think it's rather undeniable that the Authors essay was not a good-faith attempt at criticizing the peer-review process, but an attempt to discredit certain fields that they didn't like, even by their own admission. I mean, the title of the essay alone should make this very clear. Here's some quotes nonetheless: > We spent that time writing academic papers and publishing them in respected peer-reviewed journals associated with fields of scholarship loosely known as “cultural studies” or “identity studies” > As a result of this work, we have come to call these fields “grievance studies” > We undertook this project to study, understand, and expose the reality of grievance studies, which is corrupting academic research > The biggest difference between us and the scholarship we are studying by emulation is that we know we made things up. > these fields of study do not continue the important and noble liberal work of the civil rights movements; they corrupt it while trading upon their good names to keep pushing a kind of social snake oil onto a public that keeps getting sicker [Note also that this essentially an academic way of phrasing the same things you'll hear people like the far-right Stefan Molyneux (on who's show one of the authors has been a guest several times) say.] Of importance here that it's not the scientific practices that are "corrupting academia" but the fact that these fields exist at all. I'm sure you can find more if you really want to. Compare this to say the "reproducibility in Psychology" study, which unsurprisingly doesn't dedicate half of their text to talking about why the authors think psychology is bad. Don't be mistaken: The whole thing was an attempt to cash in on the general crisis the scientific process is facing to score some easy political points and not a neutral and constructive attempt at improving the quality of research in the field. Or rather, their proposal at improving the quality would involve getting rid of the fields that they don't personally like.
dang 8y ago
Please don't take HN on classic flamewar tangents. It leads to the same few hot controversies taking over everything else, and the quieter, less predictable discussions that we actually want here are the ones that tend to get burnt out of the picture as a result. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

michaelmrose 8y ago

Yes

notduncansmith 8y ago
I also believe what they did was justified. They did actual science and revealed that a number of publishers within this field demonstrate such a strong preference for virtue-signaling papers that they have lowered the bar for critical thought in both content and review. One of the honeypot papers won an award. That shows you how obvious the publisher bias is, and how obvious it must have been to these people before they decided to do something about it. Not only did these individuals risk their careers to help save a dying system with this project, they held themselves to a pretty high standard of execution. We place lots of limits on the amount of heroism individuals are allowed to display, and I think that’s mostly beneficial - but sometimes we need heroes.

gizmo686 8y ago

I'm not an academic, but aren't there standard practices for doing that sort of ethically questionable research. Namely, an ethics committee that you go to for the green light before proceeding.

learc83 8y ago
There's usually an institutional review board that you go through when using research subjects, but you don't need to do that to publish something like a new algorithm you developed in a CS journal. In this case, the researcher probably should have gotten IRB approval because there were test subjects--the journal editors, peer reviewers, and probably journal readers. But I'm assuming they just skipped that process and said there were no human subjects.

Baeocystin 8y ago

>In a way I can understand this; they did after all submit papers they knew weren't academic (in bad faith?) in order to prove a greater point. But does that justify what they did? A thousand times yes. Exposing cargo cults that attempt to co-opt the hard-earned cachet of genuine scientific inquiry for their own ideological ends is exactly what is needed to keep science functional.

lapnitnelav 8y ago
Yeah but it's a thin line to walk. If you don't do it, then you let all the pseudo-science out there masquerading as real science, which is bad. But if you overdo it, you feed into the distrust of proper science too, see Brexit and we're done listening to experts soundbite, or any of the anti-vax crowd.
- Baeocystin 8y ago
  In my experience, people that have profoundly anti-scientific beliefs have always been such; whether their purported trigger is Jenny McCarthy or an odd alignment of Jupiter, it's hard to know if it's even relevant- the problem is that they don't have the skillset to make rational choices. And I just don't see a Sokal-style hoax making a whit of difference in their minds, as there are a million other things that could do the same. In other words, their feelings of anti-science came first. Whatever reason provided is ex post facto confabulation. Meanwhile, the sciences themselves need this kind of scrutiny to stay scientific. I'm open to being wrong. But I think it is a mistake to put too much weight to the 'we should be careful what the people who never use science anyway' consider.

shrewduser 8y ago

there are a number of fields that seem to operate on motivated reasoning over objective truth.

pasabagi 8y ago

These kind of stunts are pretty annoying, though. I know with the original (who was it again?), it was essentially a case of the academic leveraging his reputation, the editors being a bit like 'this seems kooky, but sure, you're the expert', then him turning around and pretending like he'd fooled everyone.

hnuser123456 8y ago
It was by James Lindsay, Peter Boghossian, and Helen Pluckrose. They submitted papers with fictitious names and institutions. They were not leveraging their own reputations (besides the publication's, if they were accepted, and a Richard Baldwin who lent his identity for some of them.) They spent time reviewing and citing existing work, so they could legitimately claim to be building on it, but then try to go in the most absurd directions possible, so absurd that any self-respecting academic publication ought to immediately reject it with disgust, and see what happened. 7 out of 20 such papers were accepted, 4 were published, they received 4 invitations to peer-review other papers, and 1 "gained special recognition for excellence." https://areomagazine.com/2018/10/02/academic-grievance-studies-and-the-corruption-of-scholarship/ https://areomagazine.com/2018/10/02/academic-grievance-studi...
- hyeonwho4 8y ago
  ... and they were only discovered to be frauds by the twitter feeds that mock social sciences research.
- pasabagi 8y ago
  If I published something in any of the innumerable scientific 'journals' that bombard my spam box, I could probably publish whatever I liked. That doesn't mean I can say science is methodologically flawed. The humanities, no doubt, deserve criticism, but I honestly don't see the appeal of stunts designed to 'prove' that a certain field is not worth the time of day, by people who don't even understand the field at the most basic level. How would they know? Why would they care? It strikes me as embarrassing for everybody involved- embarrassing, to publish something without properly vetting it, embarrassing, to dismiss something as nonsense without actually having the measure of it, embarrassing, to be so enthusiastic about this kind of 'proof', which so obviously combines ignorance with arrogance. There is no such thing as grievance studies. Critical theory has nothing to do with identity. The whole thing is about as close to the mark as Pat Buchanan's critique of feminism, and about as clever. In short, depressing.

pfortuny 8y ago

Of course it is justified: you can only prove that kind of abuse by making it glare: otherwise, the “secrecy” of the peer-review process is the excuse.

bartread 8y ago

> But does that justify what they did? Yes, it does. These journals are an absolute joke, make a mockery of any serious attempt to advance knowledge and understanding, and deserve to be pilloried. Their standards of rigour and review are shockingly poor and as such they do not provide a suitable vehicle for the publication of genuine research, or for measured consideration of the content they do publish. Just this week I ran across this, where a couple of the academics concerned were interviewed by Joe Rogan late last year: https://www.youtube.com/watch?v=AZZNvT1vaJg&t=1s https://www.youtube.com/watch?v=AZZNvT1vaJg&t=1s Fascinating, horrifying, and entertaining in equal measure. For more background, there's also this on wikipedia: https://en.wikipedia.org/wiki/Grievance_Studies_affair https://en.wikipedia.org/wiki/Grievance_Studies_affair

pfortuny 8y ago
Thank you for your clarity. We need more of this.

CJefferson 8y ago

It doesn't justify what they did, and what they did has very little value, for 4 reasons. 1) there are ethics panels for a reason -- many people have run unacceptable experiments on people. Academics don't get to skip the ethics checks because fhey feel like it. 2) their experiment was poorly designed. On their first attempt of their papers got ejected, so they polished and resubmitted them. What is their exact scientific theory? 3) they were clearly trying for contraversy. Maybe some part of Mein Kampf are actually useful valid statements to make. Just because of the terribleness of the Nazis, doesn't mean everything Hitler ever said, or wrote, should automatically be rejected from journals. 4) maybe Grievence Studies actually review better than everyone else. Any good study need a control.

kristianc 8y ago
> 2) their experiment was poorly designed. On their first attempt of their papers got ejected, so they polished and resubmitted them. What is their exact scientific theory? They've outlined this. There is a specific Grievance Studies nomenclature and orthodoxy which the first papers did not adhere to. When they republished and resubmitted, they followed the Orthodoxy and they got through. That IS the scientific method. You run an experiment, observe the results. You tweak a few variables and see what happens under those conditions. It’s what distinguishes their work from Grievance Scholarship, where you know the conclusion you want before you start. > Maybe some part of Mein Kampf are actually useful valid statements to make. Um?
- CJefferson 8y ago
  Let's pick a random bit of Mein Kampf (I just scrolled to a random page in the middle in a randomly chosen translation). This looks, to me, like something that wouldn't be out of place in any discussion of "Fake News". This is the problem with using bits of Mein Kampf, it feels designed to shock and offend. If I got this text below into an article in a major newspaper, would that make that newspaper fundamentally broken / untrustworthy, because it happens to come, word for word, from Mein Kampf? Quote begins: Just in journalistic circles one usually prefers to call the press a 'great power' of the State. As a matter of fact its importance is truly enormous. It cannot be overestimated ; it is indeed actually the continuation of the education of youth in advanced age. Thereby one can divide the readers as a whole into three groups: First, those who believe everything they read; Secondly, those who no longer believe anything; Thirdly, those who critically examine what they have read and judge accordingly.

ATsch 8y ago

I think it's surprising that on a site such as HN the narrative of these authors is so readily accepted without any second thought. The claim is that they just made up trash and submitted it to high-quality peer-reviewed journals and they accepted them. The reality is much less outragous. Of the 20 papers, 4 were published and 3 more accepted for publication. They spent a year doing this. The authors note that they submitted the papers to "top journals in the field", leaving out the fact that all of those top journals rejected the 20 papers and they had to revise the papers and send them to shoddier journals instead. The review quoted by the authors was from an unsure grad student doing their first review of a paper. Said student later said they knew the paper was crap, but still tried to be constructive. The assertion of the authors is that this stuff getting published means the whole field is worthless. Which is of course ridiculous. Medicine didn't become obsolete when a journal published a paper with an abstract ending in "The fact that these last sentences appear in the published paper tell you, dear reader, exactly how seriously the editorial process has been taken". Computer Science didn't implode when three MIT students made an AI to generate random papers and 120(!) were accepted by the reputable IEEE and Springer. The blame was back then correctly put on the publishers and the general flaws of the current scientific process instread. So, as people have pointed out, it's ironic that their essay itself is incredibly scientifically unrigorous. No controls, unsourced claims, biased authors and conclusions that aren't supported by the data.

prepend 8y ago
“The assertion of the authors is that this stuff getting published means the whole field is worthless.” I don’t get this at all from the authors. My take is that they call out serious flaws in the rigor of how papers are reviewed. I don’t think it’s reasonable to extend that to say the field is worthless. I did read through many papers in the referenced journals and it seems really surprising to me. I’d love to see some of the research reproduced. But I’m not even sure how you could scientifically prove a field worthless.
- ATsch 8y ago
  > I don’t get this at all from the authors. My take is that they call out serious flaws in the rigor of how papers are reviewed. I don’t think it’s reasonable to extend that to say the field is worthless. I think it's rather undeniable that the Authors essay was not a good-faith attempt at criticizing the peer-review process, but an attempt to discredit certain fields that they didn't like, even by their own admission. I mean, the title of the essay alone should make this very clear. Here's some quotes nonetheless: > We spent that time writing academic papers and publishing them in respected peer-reviewed journals associated with fields of scholarship loosely known as “cultural studies” or “identity studies” > As a result of this work, we have come to call these fields “grievance studies” > We undertook this project to study, understand, and expose the reality of grievance studies, which is corrupting academic research > The biggest difference between us and the scholarship we are studying by emulation is that we know we made things up. > these fields of study do not continue the important and noble liberal work of the civil rights movements; they corrupt it while trading upon their good names to keep pushing a kind of social snake oil onto a public that keeps getting sicker [Note also that this essentially an academic way of phrasing the same things you'll hear people like the far-right Stefan Molyneux (on who's show one of the authors has been a guest several times) say.] Of importance here that it's not the scientific practices that are "corrupting academia" but the fact that these fields exist at all. I'm sure you can find more if you really want to. Compare this to say the "reproducibility in Psychology" study, which unsurprisingly doesn't dedicate half of their text to talking about why the authors think psychology is bad. Don't be mistaken: The whole thing was an attempt to cash in on the general crisis the scientific process is facing to score some easy political points and not a neutral and constructive attempt at improving the quality of research in the field. Or rather, their proposal at improving the quality would involve getting rid of the fields that they don't personally like.

dang 8y ago

Please don't take HN on classic flamewar tangents. It leads to the same few hot controversies taking over everything else, and the quieter, less predictable discussions that we actually want here are the ones that tend to get burnt out of the picture as a result. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html