13 ms·

I'm not a big fan of this article at all. Firstly, if you want to try and actually persuade people of a point don't preface your entire discussion by deciding t

by slivym 8y ago

I'm not a big fan of this article at all. Firstly, if you want to try and actually persuade people of a point don't preface your entire discussion by deciding to use an epithet to describe what you're discussing. How can I take seriously someone who goes straight in and goes "This is grievance studies"? It seems kind of funky how meta it is to adopt that epithet and then go on to make the accusation that the people you are talking about are acting in bad faith. Reading through the methodology I'm really struggling to see what the actual value of what they're doing is. For example - the Dog Park study: >What if we write a paper saying we should train men like we do dogs—to prevent rape culture? Hence came the “Dog Park” paper. If you want to assert that this is a nutty idea - please back up that claim. Because I feel fairly comfortable arguing that we can learn a lot about social constructs by studying how they differ in other species. There's nothing wrong with that concept. Maybe there's something glaringly wrong in that particular paper, but these authors do nothing to actually demonstrate that. Later on that go into detail with this. That paper is meant to have implausible statistics. Ok, what is peer review meant to do? Reject studies that have implausible findings? We have a word for that - Bias. The correct response to a paper with implausible findings is to publish it and let people follow up and do their own studies. That is the system working correctly. As for advocating for outlandish actions as a result of the study. Well authors are welcome to suggest anything that like as a result of their research. Publishing it doesn't imply endorsement. It implies "This is a real study". So let's talk science. I have a confounding variable - bad studies got published because it's better to publish lots and let the academic community decide the merits of the study than to censor what you think is either implausible or has troubling parts. That explains all the results of this study without the wildly insulting and vaguely conspiracy theorist assertion that these papers get published because of an agenda on the part of academics.

calahad 8y ago
"Maybe there's something glaringly wrong in that particular paper, but these authors do nothing to actually demonstrate that." He first states that the particular paper's flaw is the use of implausible statistics, then goes into more detail: "There was also considerable silliness including claiming to have tactfully inspected the genitals of slightly fewer than 10,000 dogs whilst interrogating owners as to their sexuality (“Dog Park”),"
- slivym 8y ago
  Is that silly? As an academic am I meant to be familiar with the number of dogs who frequent a particular park in a country I've never been to? As the reviewer of that paper what exactly are you suggesting, I accuse them of lying? Academia is set up to tackle people who fabricate their results - the reputational damage would destroy most people's careers. But that mechanism is not some sort of fact-checking investigation by the peer reviewers. Let me put the counter to you: That paper has been cited precisely 0 times according to publications website. In fact, the only references I can find to it are non-academic websites which generally trawl research for funny papers they can write jokey articles about. So what has this told us about Academia?
  - foldr 8y ago
    >Academia is set up to tackle people who fabricate their results - the reputational damage would destroy most people's careers. But that mechanism is not some sort of fact-checking investigation by the peer reviewers. This is an important point, and slivym shouldn't be downvoted for making it. Many people outside academia seem to have unrealistic expectations of the peer review process. Reviewers can't, in most cases, verify experimental results. Ironically, this is especially true of the so-called "hard" sciences, where a typical experiment might take months or years of preparation and cost lots of money to carry out. The only real protection against fabricated results, in any field, is the honesty of its practitioners. What we have here is a group of rather naive people discovering that it's easy to lie and get away with it.
    - buboard 8y ago
      > Many people outside academia seem to have unrealistic expectations of the peer review process Disagree , and i think slivym has confused peer review with sloppy view, or perhaps peer review is indeed sloppy in their field. Yes, peer review is a bad system but it is not "nothing". Reviews won't redo the experiment but they may ask for a lot of work in reviews , they can be opinionated , disbelieving, and this is good, it keeps a certain baseline, and in my experience always improves the paper. You can easily tell if a manuscript that has been reviewed or first-submitted. It s not a matter of black and white: peer review sits somewhere in between but it's not black.
buboard 8y ago
> there's nothing wrong with that concept Indeed there is nothing wrong with any concept. If they wanted to show some systemic bias they should also publish a study about how to train women like dogs to attack men. > Reject studies that have implausible findings? Yes (if by implausible you mean unconvincing). Reject and resubmit with more powerful statistics. > or publish it and let people follow up and do their own studies Indeed , like good software, you publish a barely functioning version and let others find the bugs /s > authors are welcome to suggest anything that like as a result of their research Not really, editors will often tone you down. Otherwise we might do away with peer and editorial review. I think you are advocating for 'anything goes' publishing here. Sure but then these journals should not call themselves scientific, it's a travesty to co-opt that name to validate their existence. Which in the end proves the point the author makes. The author clearly has an agenda here, that's why they re publishing a blog, not a "science journal"
- slivym 8y ago
  >Yes (if by implausible you mean unconvincing). Reject and resubmit with more powerful statistics. I should clarify here- I mean implausible by the standard of common sense, but backed up by the statistics presented in the paper, which was the case here.
jlawson 8y ago
It's not just that they were published with "implausible "results. It's three things: 1. That they were published with obviously impossible results (inspecting the genitals of 10,000 dogs while asking owners about their sexuality). 2. It's where they were published. e.g. In the #1 "best" feminist journal Hypatia. 3. It's the evil in the messages they got published. They got a 3000-word except of Mein Kampf published, by rewording it in the language of intersectional theory. They got published recommending that "privileged" students be physically chained and silenced during class. They got published by proposing men be trained like animals. (This is not the same as learning about human behavior by studying dogs; it is to suggest that it's okay to treat people like dogs if they're born a certain way). All this demonstrates a field of study that is not just soft in terms of intellectual rigor ("implausible"), but is willing to casually support reprehensible policy ideas if they're framed within a specific neo-Marxist belief system and targeted at men or white people. Bear in mind that many of the people producing the papers in these fields receive public funds at public institutions. They project an image of being neutral scholars to justify those funds. What this hoax has done is given evidence that they are not neutral scholars. They are moralizers, activists, revolutionaries working to target a specific group of people, and are not very concerned about the rigor of the intellectual pathways they use to get there. A group of people dedicated to a process are called scientists. A group of people dedicated to a conclusion is called a religion or an ideology. Social scientists claim to be the former. But the hoax papers demonstrated that, at least with some regularity and even in the highest journals, the social sciences act like the latter.
- morsch 8y ago
  They got published recommending that "privileged" students be physically chained and silenced during class. That submission actually was not published.
- ForHackernews 8y ago
  > (inspecting the genitals of 10,000 dogs while asking owners about their sexuality). Why is that 'obviously impossible'? How long does it take to look at a dog's genitals, 5 minutes? I haven't read the paper, but you could get a team of 10 undergrad (or grad) students, and send them out to different dog parks in the area. If each student inspects 20 dogs in a day (say they go to the park for a few hours--I assume many owners will say no) then you've got your data after 50 days of work. Say your students are only free on weekends (when dog parks are likely more busy, anyway) then it'll take you a little more than six months to gather all your doggie-dick-data. That's not at all an unreasonable length to run a study.
ForHackernews 8y ago
I think it's kind of shady that these three only published very selective quotes from their reviewers. Any skeptical or critical comments (even on the papers that were rejected) are omitted. Given how long this article is, it doesn't seem likely they were omitted for length. They're trying to present a one-sided slant of the peer reviewers as overly credulous.
- thinkingemote 8y ago
  For those interested into following this bias into whether it was shady or not, all the reviews can be read on a shared Google Drive folder: https://drive.google.com/drive/folders/19tBy_fVlYIHTxxjuVMFxh4pqLHM_en18 https://drive.google.com/drive/folders/19tBy_fVlYIHTxxjuVMFx... as linked to from the article.
tptacek 8y ago
It's a little telling that a bunch of their most lurid examples from the main text turn out to have been rejected; they seem to want to spin a narrative from the nice things reviewers said in the process of rejecting those papers. For instance, the "Masturbation" paper wasn't even "reject and resubmit"; it was a straight reject. If you don't read all the way to the end, you can miss that fact easily. (Amusingly, they also print quotes from the same reviewer twice on that paper, creating the impression that multiple reviewers responded positively to it; I'm not going to take the time to audit all their peer quotes, but I'll bet that's a trick they applied repeatedly.) Another example: Hypatia is apparently quite well known in feminist studies. The authors got a paper accepted in Hypatia. The authors also suggested that Hypatia was "surprisingly warm" to a paper (actually, it turns out, an essay) that suggested that white men be forced to sit on the floor in chains in classrooms. What the write-up doesn't tell you is that that's not the paper Hypatia accepted; they accepted a much more banal paper about the legitimacy of academic hoaxes. Another example: they highlight (and so does their press coverage) a paper that spins a section of Mein Kampf into a critical race/gender theory argument. What they don't say up front is that they tried this three times, once with a relatively serious journal, once with a journal with an impact factor of 1.1, and finally with a 0.8. It's the 0.8 that accepted them; the others rejected. In fact, academic science --- STEM included --- is littered with bullshit journals, but generating that fact as the result of their "experiment" wouldn't have made for such an interesting story. Kieran Healy makes an interesting point about this on Twitter: it's probably easier to get totally fake papers accepted than real ones. Recall Matt Might's cartoon essay about PhD theses and the tiny dent any one paper makes at the boundary of a science. If you aren't constrained by reality, it's easy to generate a paper that seems impactful. Meanwhile, the whole enterprise of peer review relies on good faith: reviewers are donating their time, and for every paper published, there are many many more that had to be read and rejected.
- Tomte 8y ago
  Matt Might.
  - tptacek 8y ago
    D'oh. Thanks. Too late to edit.

by slivym 8y ago

calahad 8y ago

"Maybe there's something glaringly wrong in that particular paper, but these authors do nothing to actually demonstrate that." He first states that the particular paper's flaw is the use of implausible statistics, then goes into more detail: "There was also considerable silliness including claiming to have tactfully inspected the genitals of slightly fewer than 10,000 dogs whilst interrogating owners as to their sexuality (“Dog Park”),"

slivym 8y ago
Is that silly? As an academic am I meant to be familiar with the number of dogs who frequent a particular park in a country I've never been to? As the reviewer of that paper what exactly are you suggesting, I accuse them of lying? Academia is set up to tackle people who fabricate their results - the reputational damage would destroy most people's careers. But that mechanism is not some sort of fact-checking investigation by the peer reviewers. Let me put the counter to you: That paper has been cited precisely 0 times according to publications website. In fact, the only references I can find to it are non-academic websites which generally trawl research for funny papers they can write jokey articles about. So what has this told us about Academia?
- foldr 8y ago
  >Academia is set up to tackle people who fabricate their results - the reputational damage would destroy most people's careers. But that mechanism is not some sort of fact-checking investigation by the peer reviewers. This is an important point, and slivym shouldn't be downvoted for making it. Many people outside academia seem to have unrealistic expectations of the peer review process. Reviewers can't, in most cases, verify experimental results. Ironically, this is especially true of the so-called "hard" sciences, where a typical experiment might take months or years of preparation and cost lots of money to carry out. The only real protection against fabricated results, in any field, is the honesty of its practitioners. What we have here is a group of rather naive people discovering that it's easy to lie and get away with it.
  - buboard 8y ago
    > Many people outside academia seem to have unrealistic expectations of the peer review process Disagree , and i think slivym has confused peer review with sloppy view, or perhaps peer review is indeed sloppy in their field. Yes, peer review is a bad system but it is not "nothing". Reviews won't redo the experiment but they may ask for a lot of work in reviews , they can be opinionated , disbelieving, and this is good, it keeps a certain baseline, and in my experience always improves the paper. You can easily tell if a manuscript that has been reviewed or first-submitted. It s not a matter of black and white: peer review sits somewhere in between but it's not black.

buboard 8y ago

> there's nothing wrong with that concept Indeed there is nothing wrong with any concept. If they wanted to show some systemic bias they should also publish a study about how to train women like dogs to attack men. > Reject studies that have implausible findings? Yes (if by implausible you mean unconvincing). Reject and resubmit with more powerful statistics. > or publish it and let people follow up and do their own studies Indeed , like good software, you publish a barely functioning version and let others find the bugs /s > authors are welcome to suggest anything that like as a result of their research Not really, editors will often tone you down. Otherwise we might do away with peer and editorial review. I think you are advocating for 'anything goes' publishing here. Sure but then these journals should not call themselves scientific, it's a travesty to co-opt that name to validate their existence. Which in the end proves the point the author makes. The author clearly has an agenda here, that's why they re publishing a blog, not a "science journal"

slivym 8y ago
>Yes (if by implausible you mean unconvincing). Reject and resubmit with more powerful statistics. I should clarify here- I mean implausible by the standard of common sense, but backed up by the statistics presented in the paper, which was the case here.

jlawson 8y ago

It's not just that they were published with "implausible "results. It's three things: 1. That they were published with obviously impossible results (inspecting the genitals of 10,000 dogs while asking owners about their sexuality). 2. It's where they were published. e.g. In the #1 "best" feminist journal Hypatia. 3. It's the evil in the messages they got published. They got a 3000-word except of Mein Kampf published, by rewording it in the language of intersectional theory. They got published recommending that "privileged" students be physically chained and silenced during class. They got published by proposing men be trained like animals. (This is not the same as learning about human behavior by studying dogs; it is to suggest that it's okay to treat people like dogs if they're born a certain way). All this demonstrates a field of study that is not just soft in terms of intellectual rigor ("implausible"), but is willing to casually support reprehensible policy ideas if they're framed within a specific neo-Marxist belief system and targeted at men or white people. Bear in mind that many of the people producing the papers in these fields receive public funds at public institutions. They project an image of being neutral scholars to justify those funds. What this hoax has done is given evidence that they are not neutral scholars. They are moralizers, activists, revolutionaries working to target a specific group of people, and are not very concerned about the rigor of the intellectual pathways they use to get there. A group of people dedicated to a process are called scientists. A group of people dedicated to a conclusion is called a religion or an ideology. Social scientists claim to be the former. But the hoax papers demonstrated that, at least with some regularity and even in the highest journals, the social sciences act like the latter.

morsch 8y ago
They got published recommending that "privileged" students be physically chained and silenced during class. That submission actually was not published.
ForHackernews 8y ago
> (inspecting the genitals of 10,000 dogs while asking owners about their sexuality). Why is that 'obviously impossible'? How long does it take to look at a dog's genitals, 5 minutes? I haven't read the paper, but you could get a team of 10 undergrad (or grad) students, and send them out to different dog parks in the area. If each student inspects 20 dogs in a day (say they go to the park for a few hours--I assume many owners will say no) then you've got your data after 50 days of work. Say your students are only free on weekends (when dog parks are likely more busy, anyway) then it'll take you a little more than six months to gather all your doggie-dick-data. That's not at all an unreasonable length to run a study.

ForHackernews 8y ago

I think it's kind of shady that these three only published very selective quotes from their reviewers. Any skeptical or critical comments (even on the papers that were rejected) are omitted. Given how long this article is, it doesn't seem likely they were omitted for length. They're trying to present a one-sided slant of the peer reviewers as overly credulous.

thinkingemote 8y ago
For those interested into following this bias into whether it was shady or not, all the reviews can be read on a shared Google Drive folder: https://drive.google.com/drive/folders/19tBy_fVlYIHTxxjuVMFxh4pqLHM_en18 https://drive.google.com/drive/folders/19tBy_fVlYIHTxxjuVMFx... as linked to from the article.

tptacek 8y ago

It's a little telling that a bunch of their most lurid examples from the main text turn out to have been rejected; they seem to want to spin a narrative from the nice things reviewers said in the process of rejecting those papers. For instance, the "Masturbation" paper wasn't even "reject and resubmit"; it was a straight reject. If you don't read all the way to the end, you can miss that fact easily. (Amusingly, they also print quotes from the same reviewer twice on that paper, creating the impression that multiple reviewers responded positively to it; I'm not going to take the time to audit all their peer quotes, but I'll bet that's a trick they applied repeatedly.) Another example: Hypatia is apparently quite well known in feminist studies. The authors got a paper accepted in Hypatia. The authors also suggested that Hypatia was "surprisingly warm" to a paper (actually, it turns out, an essay) that suggested that white men be forced to sit on the floor in chains in classrooms. What the write-up doesn't tell you is that that's not the paper Hypatia accepted; they accepted a much more banal paper about the legitimacy of academic hoaxes. Another example: they highlight (and so does their press coverage) a paper that spins a section of Mein Kampf into a critical race/gender theory argument. What they don't say up front is that they tried this three times, once with a relatively serious journal, once with a journal with an impact factor of 1.1, and finally with a 0.8. It's the 0.8 that accepted them; the others rejected. In fact, academic science --- STEM included --- is littered with bullshit journals, but generating that fact as the result of their "experiment" wouldn't have made for such an interesting story. Kieran Healy makes an interesting point about this on Twitter: it's probably easier to get totally fake papers accepted than real ones. Recall Matt Might's cartoon essay about PhD theses and the tiny dent any one paper makes at the boundary of a science. If you aren't constrained by reality, it's easy to generate a paper that seems impactful. Meanwhile, the whole enterprise of peer review relies on good faith: reviewers are donating their time, and for every paper published, there are many many more that had to be read and rejected.

Tomte 8y ago
Matt Might.
- tptacek 8y ago
  D'oh. Thanks. Too late to edit.