7 ms·
> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. What's your evidence for that? A number of t
by darawk 8y ago
> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals.
What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia.
> They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper
The thesis of the paper that got accepted was that "criticizing feminist theory is unethical". That doesn't seem particularly banal to me.
> They quote almost entirely favorable feedback --- including, in their earlier writeup, from papers that were ultimately rejected!
Who cares if the feedback was on papers that were ultimately rejected? Does that change the fact that they received favorable feedback on an absurd hypothesis?
> They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal.
This is simply not true. They got accepted by high impact journals, including Hypatia. You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted.
> In at least one case, they cite the same reviewer multiple times, leaving the impression of a consensus.
Are you saying that they do not cite a diverse sample of reviewers? Because that is empirically false. What you seem to be doing is latching a single instance where they cited the same reviewer twice, and trying to use that to insinuate that they didn't cite feedback from lots of distinct individuals, which is patently false.
> 1. Peer review isn't replication and never could have been. Peer reviewers get, at best, a couple hours, unpaid; submitting researchers get many months, and are often grant-funded. The hoaxers took ten months to get a few papers accepted in marginal journals. The pop understanding of what peer review is is, simply, broken: peer review depends on good faith, and it was never realistic to believe otherwise.
Is that so? So, you'd have no problem getting a nonsense paper accepted into a math, computer science, physics or even economics journal? I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in any of these disciplines.
- gbhn 8y agoI read over a couple of the papers. The descriptions of them by the authors did not match the content of the papers. For instance, the one they call "Hooters" is described as "A gender scholar goes to Hooters to try to figure out why it exists." The actual paper is written as a record of transcripts of visits to a restaurant and an extraction of particular conversation themes. At no point did I see the paper questioning why Hooters exists -- the (apparently totally faked) data seems pretty stereotypically motivated, which I think may be their point, but I'm not sure short of a fraud investigation how reviewers are supposed to know that the particular group this author claims to have visited Hooters with didn't say the things supposedly directly recorded. That's a really dramatic claim for a reviewer to make: "This conversation which the author claims is a direct transcription from a recording couldn't possibly have happened because it seems too stereotypical." Yes, the methodology isn't that great, and the paper was rejected. More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature. Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion.
- darawk 8y ago> I read over a couple of the papers. The descriptions of them by the authors did not match the content of the papers. So did I. The dog park paper more or less matches its description. The fat bodybuilding paper is exactly as described. The dildos paper is as described. Do you have any descriptions aside from the Hooters one that you take issue with? > Yes, the methodology isn't that great, and the paper was rejected. Sure, but lots of them were not. I'm not sure why you would focus on a paper that was rejected when they had 7 that were accepted. > More generally, the "study" isn't selecting random journals to see if they could defraud -- it is instead aimed at specific academic targets and a "test to failure" scattershot mechanism is used. Thus we have no idea whether these journals are any more discriminating than, say, PLOS or Nature. True, though I have a hard time believing you could get the same result in any STEM discipline, though i'm certainly open to data to the contrary. > Ultimately, I had some sympathy for the Sokal experiment in that it seemed to say something about the interaction of literary theory and physics. I don't get the same sense that there's much here other than "confirmation bias exists, even among gender studies folks!!!" which seems like it wouldn't take 3 people 10 months to figure out, and could be done in a much more direct way, and honestly isn't that shocking or rattling a conclusion. I think the point of all this is that the epistemology of the field is intellectually bankrupt. It's not just confirmation bias. It's that within the epistemology of gender studies, they aren't even wrong.
- tomlock 8y agoCould you reference some texts from within gender studies that you feel are both representative of the field, and which outline the negative aspects of the epistemology of gender studies that you are perceiving?
- tomlock 8y ago> Is that so? So, you'd have no problem getting a nonsense paper accepted into a math, computer science, physics or even economics journal? I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in any of these disciplines. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis > A 2016 study in the journal Science found that two-thirds of 18 experimental studies from two top-tier economics journals (American Economic Review and the Quarterly Journal of Economics) successfully replicated. A 2017 study in the Economic Journal suggested that "the majority of the average effects in the empirical economics literature are exaggerated by a factor of at least 2 and at least one-third are exaggerated by a factor of 4 or more".
- darawk 8y agoThe replication crisis is an entirely different thing. I'm not sure what point you're trying to make.
- tomlock 8y agoIt's entirely possible to get a nonsense paper into an economics journal, even when not intentionally trying to expose the discipline as "nonsense". I'm not sure what you're not understanding here.
- darawk 8y agoReplication crisis != nonsense papers, i'm not sure what you're not understanding here.
- tomlock 8y agoI guess I'm not understanding how you can think that a paper that makes a claim about the real world, but doesn't apply to the real world, could be characterised as anything but nonsensical.
- 8y ago
- fenomas 8y ago> You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted. The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!". The authors here can't say anything like that. They wrote real papers, arguing real conclusions they thought the journals might agree with. Then those papers got rejected, so they toned it down and wrote more, and then they got peer-review feedback so they changed things in response, etc. The most one can fairly say about the results is "Hey look, these journals published papers that we think are really bad". Which is fair enough, but surely they could say the same thing about most of what those journals already publish, right? The liberty of the whole thing is writing it all up as if a Sokal-style "we got them to print gibberish" narrative had occurred, when that seems clearly not to be the case.
- darawk 8y ago> The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!". My point is that most of them were not banal. In that line, I was responding to an isolated point of tptaceks. If all the papers were banal, i'd agree. But they were not.
- tptacek 8y agoMost of the papers were rejected! The ones that were accepted did tend to be banal!
- darawk 8y agoSure, most got rejected. But 7 got accepted. And while banality is obviously a subjective value judgment, I don't think it applies to all of them. Just to enumerate the hypotheses that were accepted: > That dog parks are rape-condoning spaces and a place of rampant canine rape culture and systemic oppression against “the oppressed dog” through which human attitudes to both problems can be measured. This provides insight into training men out of the sexual violence and bigotry to which they are prone. > That it is only oppressive cultural norms which make society regard the building of muscle rather than fat admirable and that bodybuilding and activism on behalf of the fat could be benefited by including fat bodies displayed in non-competitive ways. > That it is suspicious that men rarely anally self-penetrate using sex toys, and that this is probably due to fear of being thought homosexual (“homohysteria”) and bigotry against trans people (transphobia). (It combines these ideas into a novel concept “transhysteria,” which was suggested by one of the paper’s peer reviewers.) Encouraging them to engage in receptive penetrative anal eroticism will decrease transphobia and increase feminist values. > That men frequent “breasturants” like Hooters because they are nostalgic for patriarchal dominance and enjoy being able to order attractive women around. The environment that breastaurants provide for facilitating this encourages men to identify sexual objectification and sexual conquest, along with masculine toughness and male dominance, with “authentic masculinity.” The data are clearly nonsense and conclusions drawn from it are unwarranted by it. (NB. One reviewer did raise concerns about the rigor of the data) > That academic hoaxes or other forms of satirical or ironic critique of social justice scholarship are unethical, characterized by ignorance and rooted in a desire to preserve privilege. > No clear thesis. A rambling poetic monologue of a bitter, divorced feminist, much of which was produced by a teenage angst poetry generator before being edited into something slightly more “realistic” which is then interspersed with self-indulgent autoethnographical reflections on female sexuality and spirituality written entirely in slightly under six hours. > That feminism which foregrounds individual choice and responsibility and female agency and strength can be countered by a feminism which unifies in solidarity around the victimhood of the most marginalized women in society. I will grant you that 'breastaurants' and the Mein Kampf one are fairly banal. The Mein Kampf paper is really only shocking if you happen to know that it is an excerpt from Mein Kampf. But the rest seem pretty non-banal to me, at least with regard to being published in an academic journal.
- tptacek 8y agoI think you've misread me. My own comment leads off by acknowledging they got a paper accepted into Hypatia --- my gripe is that it's not the lurid farce they suggest they got accepted (a paper demanding that white male students sit on the floor in chains in class) but rather a much more boring paper about the legitimacy of hoaxes like theirs. When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them. Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work).
- darawk 8y ago> My own comment leads off by acknowledging they got a paper accepted into Hypatia --- my gripe is that it's not the lurid farce they suggest they got accepted (a paper demanding that white male students sit on the floor in chains in class) but rather a much more boring paper about the legitimacy of hoaxes like theirs. IMO the paper they did get accepted is still quite disturbing. > When I said "They submit approximately the same paper to multiple journals and are rejected by all but the very most marginal", I was referring to a specific paper (in fact, the Mein Kampf paper), not to all the papers. My point was: they tried that paper in several places, and the (academically) reputable venues rejected them. Ok, but what about the others? > Yes, I think --- I'm actually pretty confident --- that I could get a false paper accepted in a CS venue. Economists have already remarked on Twitter that you don't have to get "nonsense" papers accepted yourself, because the literature is already full of what turns out to be nonsense (do keep in mind that in large part what these hoaxers submitted was not nonsense, but rather fabricated research including field work). False, sure. False papers get accepted all the time. That's part of the scientific process. These papers were not merely false, they were absurd. I seriously doubt you could get a truly absurd CS paper published anywhere remotely reputable.
- YeGoblynQueenne 8y ago>> I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in [math, computer science, physics and economics]. What about medicine, genetics and neuroscience? These are the fields investigated by John Ioannidis, whom you've probably heard of [1]. He's the guy that published that paper titled "Why most published research findings are false" [2]. From the abstract of which, I quote: Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. Note that that's pretty much the claim of the hoax- that the hoaxed fields are dominated by bias and the majority of their research claims are absurd (so, false, I guess). If that's true of medicine and genetics, it's true of most of the hard sciences, and no doubt, also CS and economics. So there is no good reason to single out gender studies or feminist philosophy for this kind of fault, or in any case, the hoaxers would have made a much more meaningful contribution if they had hoaxed a maths or physics journal, rather than the ones they did. Btw- in maths, a recent HN article discussed the Mochizuki proof of the ABC conjecture, that his fellow maths experts consider to be very badly mistaken (if I got that right) [3]. _______________________ [1] https://en.wikipedia.org/wiki/John_Ioannidis#Research_findings https://en.wikipedia.org/wiki/John_Ioannidis#Research_findin... [2] https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... [3] https://www.quantamagazine.org/titans-of-mathematics-clash-over-epic-proof-of-abc-conjecture-20180920/ https://www.quantamagazine.org/titans-of-mathematics-clash-o...
- darawk 8y agoI am familiar with all of the situations you've cited, and they are nothing like this one. You're really misunderstanding Ionnandis' work if you think it is relevant to this. He's making arguments about bad statistics, which is super important and valuable, but this has almost nothing to do with that. Again, the Mochizuiki situation is quite different. You're really not understanding these things if you think they're comparable. Mochizuiki's work is legitimate and real and serious. Whether or not it is correct is difficult to ascertain, because of its genuine complexity. The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content. Which means that their work is indistinguishable from nonsense and absurdity. The work that Ioannidis did demonstrates that people acting within a good framework are producing work with a higher error rate than you might expect given the claims they make. That work is extremely important. But it doesn't invalidate the field of economics, or medicine. This work attempts to invalidate the entire edifice that these disciplines are built upon. It is extremely different and not comparable at all.