4 ms·

For those interested in this, take a look at the work regarding "grievance studies" by Peter Boghossian, James Lindsay, and Helen Pluckrose. They made intention

For those interested in this, take a look at the work regarding "grievance studies" by Peter Boghossian, James Lindsay, and Helen Pluckrose. They made intentionally unscientific studies with the intent of publication, and actually got themselves published on a few of their works. The objective was to demonstrate that not only does academia in the social sciences operate according to a completely different standard than the hard sciences, but is so ideologically driven that they behave in much the same way as a religion might.

danieltillett 7y ago
They would have got a lot more published except their experiment was sprung by some reporter. The social sciences (along with most of the humanities) appear to be beyond saving at this point and it might be a good idea to make a hard break with the past and start again with people trained outside the field.
naasking 7y ago
I think the main takeaway from the Sokal Squared hoax was that reviewers in this field are either a) extremely biased and approve any results that agree with their beliefs, or b) anti-science, or c) scientifically illiterate, because they couldn't even spot the painfully clear methodological problems in the hoax studies that were inserted on purpose to test exactly the quality of review.
- weberc2 7y ago
  Those aren’t mutually exclusive
claudiawerner 7y ago
People (perhaps you too) seem to have the idea that only social sciences are vulnerable to publishing stings like the one Boghossian and his friends pulled off, but that's not true[0][3]. Next, all the places the "hoax" papers were [mostly] submitted to were of low prestige (i.e poor ranked)[1]. The paper they wrote as a summary of their conclusions does not even rigorously define what field(s) in the social sciences they have issues with, instead they collect it under the nebulous term "grievance studies" - which is a term they made up themselves to deride academics (so much for the scientific spirit of camaraderie[4]). The "hoax" they pulled off doesn't seem to be showing what they say it does, at least not to the degree they suggest; from here[2]: >Let’s analyze the hoax a bit more carefully. The team wrote up 21 bogus papers altogether. (The essay starts by saying there were only 20; according to Lindsay, that’s because two of the papers were largely similar to one another.) Of those 21, two-thirds never were accepted for publication. The Areo essay dwells on several papers that had been rejected outright, including one suggesting that white students should be enchained for the sake of pedagogy, and another proposing that self-pleasure could be a form of violence against women. They take it as a sign of intellectual decay that such papers managed to elicit respectful feedback from reviewers, even short of publication. Academics warn against doing what Boghossian and friends did for their own good[5]. >The hoax was cruel; it sought to discredit targeted journals by setting a trap that exploited the scholarly predispositions of their editors and reviewers. Moreover, because of the anonymous review system, once the tricksters revealed their intent, only the editors whose names appear on the journal’s masthead suffered the sting of adverse public scrutiny. More finessed responses by the disgruntled threesome would have employed tactics such as persuasion, insight, engagement with the actual scholarship, and good sportsmanship (Bergstrom, 2018).[6] Most importantly, the researchers didn't include a control group for their study. How can they claim to be outing "bad science" when their own methodology is so poor and fails to prove anything other than anecdota and suspicion? One of the reviewers subject to the hoax wrote: >"Anyways, I guess I could be more critical in the future, but I assumed a grad student had written a confusing paper and I tried to be constructive. I'm embarrassed that I took it as seriously as I did, I'm annoyed I wasted time writing a review, and I'm glad I rejected it."[8] P.Z. Meyers put the situation best[7]: >If you can find a bad article accepted for publication in a feminist journal, please do jump on it and tear it apart. That contributes to the strength of the discipline. Don’t write a bunch of bad articles of your own, which are clearly intended only to weaken the whole discipline and provide a set of easy, straw-man arguments that you can use to pretend you’re a smart guy. And for the icing on the cake: Sokal himself isn't all that impressed with Boghossian's efforts[9]. A good thread discussing the hoax is on the social sciences subreddit[10]. [0] https://www.newscientist.com/article/dn17288-crap-paper-accepted-by-journal/ https://www.newscientist.com/article/dn17288-crap-paper-acce... [1] https://i.redd.it/qsi6i5rbv3q11.png https://i.redd.it/qsi6i5rbv3q11.png [2] https://slate.com/technology/2018/10/grievance-studies-hoax-not-academic-scandal.html https://slate.com/technology/2018/10/grievance-studies-hoax-... [3] https://platofootnote.wordpress.com/2017/05/24/an-embarrassing-moment-for-the-skeptical-movement/comment-page-2/ https://platofootnote.wordpress.com/2017/05/24/an-embarrassi... [4] https://www.3quarksdaily.com/3quarksdaily/2018/10/bad-arguments-on-bad-arguments-the-sokal-squared-hoax-as-an-unfortunate-cliche.html https://www.3quarksdaily.com/3quarksdaily/2018/10/bad-argume... [5] https://www.sciencedirect.com/science/article/abs/pii/S0378378218307242 https://www.sciencedirect.com/science/article/abs/pii/S03783... [6] https://journals.sagepub.com/doi/full/10.1177/1473325019833834 https://journals.sagepub.com/doi/full/10.1177/14733250198338... [7] https://freethoughtblogs.com/pharyngula/2018/10/03/give-it-a-rest-boghossian-and-pals/ https://freethoughtblogs.com/pharyngula/2018/10/03/give-it-a... [8] https://twitter.com/dwschieber/status/1047497301021798400 https://twitter.com/dwschieber/status/1047497301021798400 [9] https://www.chronicle.com/article/What-the-Conceptual/240344 https://www.chronicle.com/article/What-the-Conceptual/240344 [10] https://www.reddit.com/r/AskSocialScience/comments/9noxmp/is_this_study_accurately_portraying_evidence_of/ https://www.reddit.com/r/AskSocialScience/comments/9noxmp/is...
- vipref 7y ago
  >> Most importantly, the researchers didn't include a control group for their study. I'm not sure what qualifies as a "good methodology"?
- RenThraysk 7y ago
  It did prove that the only people capable of pointing out there were problems with the hoax papers are people generally critical of the social sciences. That alone demonstrates a problem. That's why the experiment ended early. PZ Meyers is a doxing buffoon.
  - claudiawerner 7y ago
    How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote): > Sure, the criticism this hoax is trying to demonstrate may be legitimate, but the methodology is one designed to highlight the cleverness of its executors and diminish the credibility of a discipline, rather than point out constructive areas for improvement. Basically, it is a methodology that does not treat its targets as intellectual equals and is quite indecorous—you get the sense that a major point of this operation is to discredit the field and make its practitioners feel some kind of public humiliation or shame. A childish tactic. [0] https://science.sciencemag.org/content/342/6154/60.full https://science.sciencemag.org/content/342/6154/60.full [1] https://news.ycombinator.com/item?id=18899529 https://news.ycombinator.com/item?id=18899529
    - arkades 7y ago
      Because the Medknow journals are notorious scam journals that will publish absolutely anything at all, without even a semblance of peer review? There's a difference between non-prestige journal and vanity press.
    - RenThraysk 7y ago
specialist 7y ago
Their "study" reminds me of Penn & Teller getting people to sign a petition banning dihydrogen monoxide (aka water). https://www.youtube.com/watch?v=yi3erdgVVTw https://www.youtube.com/watch?v=yi3erdgVVTw Just another form of social engineering. Haha, we compiled footage from a bunch of randos to show how stupid people. Haha, aren't we smart! It's the academic equivalent of tricking someone into walking into a punch.

danieltillett 7y ago

They would have got a lot more published except their experiment was sprung by some reporter. The social sciences (along with most of the humanities) appear to be beyond saving at this point and it might be a good idea to make a hard break with the past and start again with people trained outside the field.

naasking 7y ago

I think the main takeaway from the Sokal Squared hoax was that reviewers in this field are either a) extremely biased and approve any results that agree with their beliefs, or b) anti-science, or c) scientifically illiterate, because they couldn't even spot the painfully clear methodological problems in the hoax studies that were inserted on purpose to test exactly the quality of review.

weberc2 7y ago
Those aren’t mutually exclusive

claudiawerner 7y ago

People (perhaps you too) seem to have the idea that only social sciences are vulnerable to publishing stings like the one Boghossian and his friends pulled off, but that's not true[0][3]. Next, all the places the "hoax" papers were [mostly] submitted to were of low prestige (i.e poor ranked)[1]. The paper they wrote as a summary of their conclusions does not even rigorously define what field(s) in the social sciences they have issues with, instead they collect it under the nebulous term "grievance studies" - which is a term they made up themselves to deride academics (so much for the scientific spirit of camaraderie[4]). The "hoax" they pulled off doesn't seem to be showing what they say it does, at least not to the degree they suggest; from here[2]: >Let’s analyze the hoax a bit more carefully. The team wrote up 21 bogus papers altogether. (The essay starts by saying there were only 20; according to Lindsay, that’s because two of the papers were largely similar to one another.) Of those 21, two-thirds never were accepted for publication. The Areo essay dwells on several papers that had been rejected outright, including one suggesting that white students should be enchained for the sake of pedagogy, and another proposing that self-pleasure could be a form of violence against women. They take it as a sign of intellectual decay that such papers managed to elicit respectful feedback from reviewers, even short of publication. Academics warn against doing what Boghossian and friends did for their own good[5]. >The hoax was cruel; it sought to discredit targeted journals by setting a trap that exploited the scholarly predispositions of their editors and reviewers. Moreover, because of the anonymous review system, once the tricksters revealed their intent, only the editors whose names appear on the journal’s masthead suffered the sting of adverse public scrutiny. More finessed responses by the disgruntled threesome would have employed tactics such as persuasion, insight, engagement with the actual scholarship, and good sportsmanship (Bergstrom, 2018).[6] Most importantly, the researchers didn't include a control group for their study. How can they claim to be outing "bad science" when their own methodology is so poor and fails to prove anything other than anecdota and suspicion? One of the reviewers subject to the hoax wrote: >"Anyways, I guess I could be more critical in the future, but I assumed a grad student had written a confusing paper and I tried to be constructive. I'm embarrassed that I took it as seriously as I did, I'm annoyed I wasted time writing a review, and I'm glad I rejected it."[8] P.Z. Meyers put the situation best[7]: >If you can find a bad article accepted for publication in a feminist journal, please do jump on it and tear it apart. That contributes to the strength of the discipline. Don’t write a bunch of bad articles of your own, which are clearly intended only to weaken the whole discipline and provide a set of easy, straw-man arguments that you can use to pretend you’re a smart guy. And for the icing on the cake: Sokal himself isn't all that impressed with Boghossian's efforts[9]. A good thread discussing the hoax is on the social sciences subreddit[10]. [0] https://www.newscientist.com/article/dn17288-crap-paper-accepted-by-journal/ https://www.newscientist.com/article/dn17288-crap-paper-acce... [1] https://i.redd.it/qsi6i5rbv3q11.png https://i.redd.it/qsi6i5rbv3q11.png [2] https://slate.com/technology/2018/10/grievance-studies-hoax-not-academic-scandal.html https://slate.com/technology/2018/10/grievance-studies-hoax-... [3] https://platofootnote.wordpress.com/2017/05/24/an-embarrassing-moment-for-the-skeptical-movement/comment-page-2/ https://platofootnote.wordpress.com/2017/05/24/an-embarrassi... [4] https://www.3quarksdaily.com/3quarksdaily/2018/10/bad-arguments-on-bad-arguments-the-sokal-squared-hoax-as-an-unfortunate-cliche.html https://www.3quarksdaily.com/3quarksdaily/2018/10/bad-argume... [5] https://www.sciencedirect.com/science/article/abs/pii/S0378378218307242 https://www.sciencedirect.com/science/article/abs/pii/S03783... [6] https://journals.sagepub.com/doi/full/10.1177/1473325019833834 https://journals.sagepub.com/doi/full/10.1177/14733250198338... [7] https://freethoughtblogs.com/pharyngula/2018/10/03/give-it-a-rest-boghossian-and-pals/ https://freethoughtblogs.com/pharyngula/2018/10/03/give-it-a... [8] https://twitter.com/dwschieber/status/1047497301021798400 https://twitter.com/dwschieber/status/1047497301021798400 [9] https://www.chronicle.com/article/What-the-Conceptual/240344 https://www.chronicle.com/article/What-the-Conceptual/240344 [10] https://www.reddit.com/r/AskSocialScience/comments/9noxmp/is_this_study_accurately_portraying_evidence_of/ https://www.reddit.com/r/AskSocialScience/comments/9noxmp/is...

vipref 7y ago
>> Most importantly, the researchers didn't include a control group for their study. I'm not sure what qualifies as a "good methodology"?
RenThraysk 7y ago
It did prove that the only people capable of pointing out there were problems with the hoax papers are people generally critical of the social sciences. That alone demonstrates a problem. That's why the experiment ended early. PZ Meyers is a doxing buffoon.
- claudiawerner 7y ago
  How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote): > Sure, the criticism this hoax is trying to demonstrate may be legitimate, but the methodology is one designed to highlight the cleverness of its executors and diminish the credibility of a discipline, rather than point out constructive areas for improvement. Basically, it is a methodology that does not treat its targets as intellectual equals and is quite indecorous—you get the sense that a major point of this operation is to discredit the field and make its practitioners feel some kind of public humiliation or shame. A childish tactic. [0] https://science.sciencemag.org/content/342/6154/60.full https://science.sciencemag.org/content/342/6154/60.full [1] https://news.ycombinator.com/item?id=18899529 https://news.ycombinator.com/item?id=18899529
  - arkades 7y ago
    Because the Medknow journals are notorious scam journals that will publish absolutely anything at all, without even a semblance of peer review? There's a difference between non-prestige journal and vanity press.
  - RenThraysk 7y ago

specialist 7y ago

Their "study" reminds me of Penn & Teller getting people to sign a petition banning dihydrogen monoxide (aka water). https://www.youtube.com/watch?v=yi3erdgVVTw https://www.youtube.com/watch?v=yi3erdgVVTw Just another form of social engineering. Haha, we compiled footage from a bunch of randos to show how stupid people. Haha, aren't we smart! It's the academic equivalent of tricking someone into walking into a punch.