6 ms·
What an Audacious Hoax Reveals About Academia
- dang 8y agohttps://news.ycombinator.com/item?id=18134681 https://news.ycombinator.com/item?id=18134681
- zwieback 8y agoNot surprising that these kind of things happen in social studies and it's easy to make fun of. However, lots of junk is also published in science and engineering, it's just not as egregious and apparent.
- jjgreen 8y agoIndeed, "audacious" is not word I'd use for these kind of games, "lazy", "shooting fish in a barrel" would be closer.
- glormph 8y agoAlso, that junk is not as likely to trickle down to the news and the broader population.
- mdarens 8y agoyeah, it's not limited to social sciences, but it gets more media attention when it is because it feeds the narrative that the humanities are useless: https://www.nature.com/news/publishers-withdraw-more-than-120-gibberish-papers-1.14763 https://www.nature.com/news/publishers-withdraw-more-than-12...
- jf- 8y agoEngineering and science have an advantage, however, in that they are empirical and readily tested. Things evaluate to true or false, even if doing the evaluation is difficult. In the humanities, things often come down to value judgements and mere assertions, which cannot be validated one way or another.
- Moodles 8y agonot to mention a ridiculously high percentage of humanities papers are never cited once. Never. Cited. Once. http://blogs.lse.ac.uk/impactofsocialsciences/2014/04/23/academic-papers-citation-rates-remler/ http://blogs.lse.ac.uk/impactofsocialsciences/2014/04/23/aca... Who can see this and not admit the field is in crisis?
- Balgair 8y agoDepends on what you mean by crisis. If you have a student that really needs to graduate soon, this condition is a boon.
- fipple 8y agoAt least there's a standard for junk science -- it replicates or it doesn't.
- hannob 8y agoit obviously skips the third option, which is the standard in most fields: nobody ever tries to replicate it.
- jf- 8y agoPlenty of people try to replicate, in trying to build on work from other labs. What they don’t do is publish when replication fails, but the failure to replicate usually becomes known in the community.
- daveFNbuck 8y agoIf people don't publish failures to replicate, how do you know how frequently replication failures become known in the community?
- jf- 8y agoThe same way you know that you walked out of your front door this morning without it having to be published in a scientific journal.
- deleted 8y ago[deleted]
- daveFNbuck 8y agoTo be clear, my question is how you know how many failed replications there are that you never find out about.
- conception 8y agoThe 2nd part of academia that isn't being talked about here is citations. Getting published can be huge, but if your paper is actually garbage no one is going to cite it. That's a far better measure of the impact of a paper - who's using the data in it.
- mturmon 8y agoWitness this classic complaint from Peter Elias: https://oikosjournal.files.wordpress.com/2011/09/elias1958ire-editorial.pdf https://oikosjournal.files.wordpress.com/2011/09/elias1958ir... published in 1958 when he was editor of the IRE (now IEEE) Transactions on Information Theory.
- zwieback 8y agoThanks, that's gold, apparently things don't change as much or fast as we think they do.
- skybrian 8y agoWhy do you say "social studies" when the sociology journals didn't fall for it?
- daveFNbuck 8y agoThere are much more egregious examples in computer science publishing. A paper that just repeated the words "get me off your fucking mailing list" over and over [1] was published in a journal. I also remember there being some automatic paper generators that resulted in several publications several years back. This is 100% an issue with bogus for-profit publishing, not with any particular area of study. [1] http://www.scs.stanford.edu/~dm/home/papers/remove.pdf http://www.scs.stanford.edu/~dm/home/papers/remove.pdf
- airstrike 8y agoFigure 1 really drives the point home
- throwaway2048 8y agoNot a journal of anybody cares about in any way whatsoever, just a scam journal that attempts to extract fees from desperate to publish researchers. Which is why they were spamming people.
- daveFNbuck 8y agoThe same is true of the journals that published the hoax papers mentioned in the article.
- TangoTrotFox 8y agoI find it phenomenally interesting that people simply say what they want to be true with no consideration for whether it is or not. We are indeed moving into a post-fact world aren't we? [1] Here is the wiki page on the 'get me off your mailing list' paper. Here [2] is the page on the journal that chose to accept (after revisions) the Mein Kampf paper. You might notice certain differences. The other journals that accepted the fake papers are part of the T&F/Springer publishing networks, and some responded in this [3] article. They were not scam journals by any means of the word, just journals blinded by ideological bias. [1] - https://en.wikipedia.org/wiki/International_Journal_of_Advanced_Computer_Technology https://en.wikipedia.org/wiki/International_Journal_of_Advan... [2] - https://en.wikipedia.org/wiki/Affilia https://en.wikipedia.org/wiki/Affilia [3] - https://reason.com/blog/2018/10/03/dog-rape-hoax-papers-pluckrose-lindsay https://reason.com/blog/2018/10/03/dog-rape-hoax-papers-pluc...
- Nasrudith 8y agoThe real meat depends on the publisher. There are always publication mills that accept basically anything or push objectively wrong agendas like AIDS denialism. It is when the bellwethers publish absolute crap that it is a sign of true rot in the field and not just a publication.
- chippy 8y agoI am slightly cautious that this hoax is itself a hoax and no actual hoax papers were published by actual academic journals.
- 21 8y agoUnless you don't consider "Gender, Place & Culture" an actual academic journal, it did happen: https://www.tandfonline.com/doi/abs/10.1080/0966369X.2018.1475346 https://www.tandfonline.com/doi/abs/10.1080/0966369X.2018.14... https://en.wikipedia.org/wiki/Gender,_Place_%26_Culture https://en.wikipedia.org/wiki/Gender,_Place_%26_Culture
- conception 8y agoJust as a comparison for folks - Gender, Place and Culture's impact factor is about 1. Nature, one of the most impactful, is 40. https://en.wikipedia.org/wiki/Impact_factor https://en.wikipedia.org/wiki/Impact_factor
- 394549 8y ago> I am slightly cautious that this hoax is itself a hoax and no actual hoax papers were published by actual academic journals. This is apparently the original account of the hoax. They had to call it off prematurely because they started getting media attention from places like the Wall Street Journal. I think the hoax is real.
- Eridrus 8y agoI think there is a real disconnect between what peer review is and what the public seems to think peer review is. The public seems to think that peer review indicates that something is true, but that's clearly not the case because we have no way of verifying any of the data used, etc. The paper that got published in a top journal "Human Reaction to Rape Culture and Queer Performativity at Urban Dog Parks in Portland, Oregon", while silly, doesn't seem like it has to be rejected if it were an honest paper that actually did the experiments they said they did. It seems no more silly than the lobster nonsense Jordan Peterson peddles and conservatives eat up. I don't personally think it's a useful paper, but most individual papers rarely are, even in CS.
- espeed 8y agoYou mean this paper by E. A. Kravitz at Harvard Medical School Department of Neurobiology that was published in the Journal of Comparative Physiology in 1999? Serotonin and aggression: insights gained from a lobster model system and speculations on the role of amine neurons in a complex behavior [pdf] https://caspar.bgsu.edu/~huberlab/private/KingSolPub.pdf https://caspar.bgsu.edu/~huberlab/private/KingSolPub.pdf
- thrower123 8y agoI wonder if many of the people so acidly contemptuous of Peterson have ever bothered to actually read what he actually writes, or just viscerally react to the hot take media coverage. He's eminently reasonable, not an alt-right troll like Alex Jones or Ben Shapiro.
- espeed 8y agoPeterson was a Harvard professor from 1993–1998 so he was there at the time this paper was being written. Now that he's at Toronto, I'm curious to know how much interaction he has with Geoffrey Hinton. He's made references to research related to the AI embodiment problem, and one of his business partners is Daniel Higgins who was at MIT.
- novembermike 8y ago
- apricot 8y agoRelevant xkcd: https://xkcd.com/451/ https://xkcd.com/451/
- deleted 8y ago[deleted]
- nostromo 8y agoThis article doesn’t mention the most ridiculous example: Affilia, a women’s studies journal, published a rewritten chapter from “Mein Kampf.” https://www.google.com/amp/s/www.wsj.com/amp/articles/fake-news-comes-to-academia-1538520950 https://www.google.com/amp/s/www.wsj.com/amp/articles/fake-n...
- 21 8y agoI think the most ridiculous example was the one where they advocated for "privileged students" (ie: white men) to sit on the floor chained in classes and not be allowed to speak, so that they can "experience reparations": > I wonder if that ‘progressive stack’ in the news could be written into a paper that says white males in college shouldn’t be allowed to speak in class (or have their emails answered by the instructor), and, for good measure, be asked to sit in the floor in chains so they can ‘experience reparations. That was our “Progressive Stack” paper. The answer seems to be yes, and feminist philosophy titan Hypatia has been surprisingly warm to it.
- rubbingalcohol 8y agoThat was mentioned in the Atlantic article, though they didn't mention that the journal rejected it conditionally because the proposed "reparations" of sitting chained on the floor and being spoken over weren't harsh enough for privileged students -- the author was offered acceptance upon revision.
- moreoutput 8y agoGeez.
- skrebbel 8y agoSource?
- mistermann 8y agoNot proof, but more commentary: https://quillette.com/2018/10/01/the-grievance-studies-scandal-five-academics-respond/ https://quillette.com/2018/10/01/the-grievance-studies-scand...
- epx 8y agoSocial Sciences are the quintessential BS job generator. It serves a purpose but not the one it thinks it does.
- dmix 8y ago> It serves a purpose but not the one it thinks it does. Pushing political agendas and keeping forever-students employed?
- epx 8y agoSomewhere around that. There is an "Angst" in society, deep inside we all know that there isn't a sunny place for everyone, while in the surface we pretend we can do it like the Baby Boomers. Making people SJWing around dog rape is a dummy load that keep them from demanding structural changes.
- Blackthorn 8y agoIs there a full list of journals that published these? This sort of hoax seems to happen every few years, but often to journals that will explicitly accept anything and let anyone who wants to give a talk, so long as it's related to the journal's mission (which means there's nothing alarmist about it).
- perpetualcrayon 8y agoI think there are a number of variables that should be controlled for on probably separate studies. 1) how likely can renowned academic get published with gibberish 2) compare #1 to how likely can an unknown get published with gibberish. 3) how likely can an unknown get a serious paper published when it was actually the renowned academic who wrote it. 4) compare #3 to how likely a renowned academic can get a serious paper published when it was actually the unknown / lesser known peer who wrote it.
- bjt2n3904 8y agoA common response to this thread is "the review process must not have looked closely enough". This tweet[1] contains screenshots of the review process of the dog park paper, and claims they received constructive feedback. As originally written, the journal wouldn't publish it. https://twitter.com/RealPeerReview/status/1047813753180438528 https://twitter.com/RealPeerReview/status/104781375318043852...
- pdpi 8y agoAlways fun to see that the Sokal Affair is reproducible science :). (https://en.wikipedia.org/wiki/Sokal_affair https://en.wikipedia.org/wiki/Sokal_affair) Edit: Hah — interestingly enough, The Atlantic refers to this as "the new Sokal Hoax" in the <title> tag and the URL, but as "an Audacious Hoax" in the page proper.
- VikingCoder 8y agoJournals are broken. A lot of people recognize that. Peer review is also broken. A lot of people recognize that. The fact that we don't publish negative results is a tragedy. We're losing so much collective knowledge that way. We learn by failing, but somehow we're ashamed to fail in public in academia. I personally hope that arXiv and things like it keep taking off. And that we end up funding respected people to review articles publicly. And that we end up funding people to try to replicate interesting results, and publish their findings. And that the news media learns to explain to people what p scores are, and what p-hacking is, and what the half-life of knowledge is, and that one new research article with p=0.05 doesn't mean jack squat. Basically the news media shouldn't cover leading research, it's just too noisy and it's difficult to understand the context and relevance of new research.
- schizoidboy 8y agoI think that's a good summary. My favorite relevant quote from Dr. John Ioannidis: > “Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.” https://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/ https://www.theatlantic.com/magazine/archive/2010/11/lies-da...
- charlysl 8y agoOther means superior to the natural sciences exist to extract alternative knowledges about stars and enriching astronomy ... incorporation of mythological narratives and modern feminist analysis of them, feminist interpretative dance (especially with regard to the movements of the stars and their astrological significance), and direct application of feminist and postcolonial discourses I cracked here.
- scottlegrand2 8y agoI think anyone that was triggered by this experiment has self-identified themselves as part of the problem.
- deleted 8y ago[deleted]
- SubiculumCode 8y agoLet us see some absolute gibberish paper get published in a solid SCIENCE journal (e.g. Impact Factor > 5) before we all go "PEER REVIEW IS BROKEN ARGHH!!"