7 ms·
Publishers withdraw more than 120 gibberish papers
- mjn 13y agoIf I'm dreaming, the ideal outcome of this would be that Springer and IEEE decide to actually apply some kind of scrutiny to the conferences whose proceedings they publish under their names. It's a pretty poorly kept secret that, while they publish some stuff that's top work in several fields, they also publish some total crap that is so bad it's inconceivable someone actually competent has spent even 20 minutes vetting it. But they seem not to care a lot. They (especially the IEEE) are coasting on a name produced through some good work over the years. Much of that reputation is well deserved, but you can only shove so much junk into IEEE Xplore before it loses the cachet it built up in previous years. If they don't shape up, it will reduce the attractiveness of IEEE and Springer for people publishing actually good work as well. Why affiliate your legitimate conference with publishers who have no quality control? The IEEE in particular is a nonprofit organization which is supposed to have a mission to advance engineering, so it's particularly problematic that it wouldn't be doing a better job.
- p4bl0 13y ago> If I'm dreaming, the ideal outcome of this would be that Springer and IEEE decide to actually apply some kind of scrutiny to the conferences whose proceedings they publish under their names. I'm sorry to break it to you but that won't happen. It would totally destroy the awfully good business model they have, which is one of the best engineered fraud ever. Right now the situation is: - Researchers, oh which almost 100% are paid by public money, do research and write their results in papers which they submit to journals / conferences / workshops. - If the targeted journal / conference / workshop want to do peer review, their editors / committee is in charge of distributing the load, those people also are researchers (also close to 100% paid by public money). - They offload the reviewing work to other researchers who hopefully are competent on the subject of the papers. Again, public money since the reviews are seen (and I think it's normal) as a normal part of the researchers' job. - The selected papers are given to the publishers and often their copyright too. - The publishers now put the papers online, put a paywall it front of them, and make private profits $$$$. - On top of that, not many people do buy individual papers, most papers are rented by academic institutions for colossal amount of money (again, public money!). I'm really not kidding when I say "colossal".
- mjn 13y agoI can see that narrative with Springer, but the IEEE is a nonprofit, the main professional society for large areas of computer & electrical engineering. It's not necessarily the best-run nonprofit, but they aren't trying to pump up a stock price or give an ROI to investors or anything. They do need some revenue to pay for their ongoing operations, but in theory they should also be very interested in ensuring that revenue-generating operations don't compromise their main mission and reputation (which is the reason to raise revenue in the first place). I think part of it is that historically the IEEE has wanted to be relatively open to new fields of research, so if you have a few professors willing to vouch for a new conference, the IEEE is willing to sponsor it and publish the proceedings. It's quite decentralized and any of the various "societies" and "interest groups" within the IEEE can sponsor a conference without central approval... and it's also fairly easy to start a new interest group. That system depends on most active IEEE members being legitimate researchers, though, who are in good faith trying to put on real conferences. If that's true, it makes sense for the approval policy to be relatively permissive. But some people have been taking advantage of this and producing vanity conferences, so this system probably needs to be tightened. Either the IEEE needs to exercise more control over conference approval, or they need to be more diligent in kicking out people who abuse the trust of the permissive system. The ACM (the main professional society for computer scientists) is somewhat more conservative about sponsoring new conferences, though that does occasionally produce the opposite problem where it takes a lot of legwork (and sometimes politicking) to get a new one accepted.
- p4bl0 13y agoI sincerely hope you are right. However, as pointed out in other comments, most researchers know what are the good and bad venues in their particular fields. This means that in reality, who is the publisher of a journal or of the proceedings of a conference matters less than the reputation of the conference or journal itself (sadly, that may be less true when it comes to recruitment in academia) which is given by the quality of the papers published in there. If I had to chose fights I guess it would be for the generalization gold open access and for the end of bibliometrics before trying to get these publishers more involved in what they publish (since we don't really need them anymore).
- chimeracoder 13y agoThis is quite the deja vu: http://www.physics.nyu.edu/faculty/sokal/ http://www.physics.nyu.edu/faculty/sokal/ As I remembered it, their "fake" (automatically generated) paper was accepted into a conference. They then tried to submit a second paper that was also auto-generated, and that's when they got caught. I may be confusing the two stories, though. In any case, it's very easy to generate your own postmodern gibberish papers - every time you visit this page, it refreshes with a new one: http://www.elsewhere.org/pomo/ http://www.elsewhere.org/pomo/
- mjn 13y agoSokal's paper was carefully hand-written, so a somewhat different situation. His goal was to show that if you used scientific jargon in a highly metaphorical way that flattered the ideological views of the editors but was pretty hilariously shallow (e.g. casually analogizing the axiom of choice in mathematics to pro-choice politics), it could get accepted. He didn't aim to show that you could auto-generate such a paper, though, merely that a human could construct one. Unlike the papers here, Sokal's paper is also not really gibberish. It's well-written and coherent in a certain surface-level sense, it's just that what it's arguing is nonsense. It strings together coherent paragraphs out of coherent sentences, in an argument that goes somewhere and which you can follow... but the argument itself is ridiculous, put together out of hilarious non sequiturs and puns masquerading as reasoning. The SciGen papers are gibberish on a much more literal level, in that their paragraphs don't actually make sense, their abstracts don't match the subject in their titles, etc.
- chimeracoder 13y agoHm, yeah, I'm definitely confusing them. In any case, I saw this headline and thought, "where is the [2004] tag?", because it was literally exactly the way my friend told me the SCIGen and/or Sokal story back then. Quite a shock to see that this was 2014.
- catmanjan 13y agoBuffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo? http://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo http://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo...
- dredmorbius 13y agoSometimes, but at others: Buffalo buffalo buffalo Buffalo buffalo Buffalo buffalo buffalo. [1] Similarly, buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo buffalo buffalo. [2] ________________________________ Notes: 1. C n v C n C n v 2. n v C n v v C n n v
- maxtaco 13y agoThe papers in question were generated by the SCIGen paper generator. Jeremy Stribling, Dan Aguayo and I are the authors of that system, with Jeremy doing the lionshare of the work. The inspiration for SCIgen was TheSpark.com's high school paper generator, which ran from about 1999 to 2001. To call these autogenerated works gibberish is an affront to those of us who hand-assembled their context-free grammars!
- jeremyjh 13y agoThis is great work. I'm curious about your motivations. Do you consider it a validation of your work that so many papers were published?
- maxtaco 13y agoAt the time, there was an arms race within the systems community to see who could flip a bigger bird to the organizers of the SCI spamference. David Mazieres and Eddie Kohler started it off when they submitted "Get Me Off Your Fucking Mailing List" [1] for publication. We wanted to one-up them and figured we could do so by submitting a random paper that might be accepted into the conference, allowing us to give an in-person talk. The plan worked up to a point, but when we asked the internet for $3k to send us to Orlando, the press and then the organizers of SCI caught wind and revoked the random paper they had previously accepted. Everything since has been gravy. We're pumped that SCIgen is still a useful weapon against charlatans the world over (this now includes you, IEEE and Springer). [1] http://www.scs.stanford.edu/~dm/home/papers/remove.pdf http://www.scs.stanford.edu/~dm/home/papers/remove.pdf
- JonnieCache 13y agoThat link is priceless. What does it remind me of? I have a half-memory of a fake scientific paper full of nonsense words and incomprehensible diagrams that was very amusing. It was a bit like this: http://img262.imageshack.us/img262/9581/waltz19pq.jpg http://img262.imageshack.us/img262/9581/waltz19pq.jpg Anyone know what I'm on about?
- blisterpeanuts 13y agoIt may not be long before passable high school and college essays can be undetectably generated by software. I wonder what we'll do then. At some point, I guess we'll have writer software that pretty much generates nicely articulated narrative for whatever it is we wish to say. "Write up something about global warming. Make it borderline skeptical, quote the Concerned Scientists and so forth. You know what to say."
- harryh 13y agoThat would mean that a computer had passed the Turing test which would be...quite an achievement.
- tokenadult 13y agoIt may not be long before passable high school and college essays can be undetectably generated by software. I wonder what we'll do then. If we fed such a program good-quality reference books, we could massively improve the quality of most Wikipedia articles.
- maxander 13y agoHappily (?) high school and college essays (and probably potential scientific publications) will also be read chiefly by software in the near future, just as GRE essays are graded in part by software today. Whichever side has the best algorithm wins.
- davidgerard 13y ago
- beloch 13y agoConference proceedings (not to be confused with a handful of actual journals that just contain "proceedings" in their name) can be thought of like the paper programs you might be handed at a play. They're usually meant to be an accompaniment to the event rather than a stand-alone publication. Their intent is to collect contributions from conference participants into a handy reference that attendees can use if they want more information, missed a talk they were interested in, etc.. Extremely high quality proceedings are peer-reviewed and might eventually evolve into a text-book. This level of quality is relatively rare though. Most conference proceedings are not peer-reviewed and many aren't even read once by the conference organizers/"editors". This usually isn't thought to be a big deal because it is assumed that, were a submission truly fraudulent, the submitter would inevitably be outed when they give their talk. However, some conferences make submitting an original paper for their proceedings mandatory for acceptance. Getting funding to attend a conference is usually a lot easier if you're presenting, so researchers are motivated to speak even if they merely wish to attend. Speaking looks better on your CV to boot. Since conference proceedings are typically ignored, researchers generally won't put their best effort into these papers. When dead trees ruled, conference proceedings would be rapidly forgotten simply because you'd have a deuce of a time finding them if you somehow heard about a paper in it that you wanted to read! They simply weren't distributed to anyone besides conference attendees and maybe a library or two. Today, electronic copies have brought poor quality conference proceedings to a far vaster audience than they were ever intended for! I'm disappointed, but not the least bit surprised that some people would submit something created by a paper generator to a conference. The blame for this falls squarely on conference organizers for requiring paper submissions but being too lazy to even read them once. If you don't want to read a stack of badly written papers, require abstracts only and, if you're still gung-ho to make some proceedings, welcome voluntary paper submissions. This incident should not shake confidence in peer-reviewed journals one iota. It might, however, discourage conference organizers from requiring papers from speakers if they're not committed to reading them all.
- hueving 13y ago>Since conference proceedings are typically ignored researchers generally won't put their best effort into these papers. It's interesting, because this is completely the opposite in many CS fields (e.g. security). The top conferences carry much more weight than any journal. It's probably related to how fast the field changes.
- ChuckMcM 13y agoI tried to read the papers but they cost $35.95 each to download.
- shasta 13y agoIt's a cruel world.
- hueving 13y agoIt's PREMIUM gibberish. :-)
- ScottBurson 13y agoThere are a couple of examples here: http://pdos.csail.mit.edu/scigen/#examples http://pdos.csail.mit.edu/scigen/#examples I found them side-splittingly funny, at least for a page or so.
- einhverfr 13y ago"Cryptographers must usually enable the partition table." These are actually really really funny.
- vesinisa 13y agoPreviously, I have understood this was a concern mostly with the so-called "open-access" journals. In 2013, the Science Magazine went to some extreme lengths to discredit these journals, that more often than not seem to lack any form of scientific peer review. A fake paper, published by a made-up person working for a non-existent university, purporting to describe a cancer drug extracted from lichen, had experimental flaws that should be blatantly obvious to even an undergraduate of the field. Of the 304 papers submitted, 157 were accepted and only 98 were rejected, with the rest not eliciting a timely reply. Only 36 papers received reviewer comments about the scientific shortcomings, and in 16 of those cases the paper was accepted anyway. Source: http://www.sciencemag.org/content/342/6154/60.full http://www.sciencemag.org/content/342/6154/60.full It seems pretty shocking that even well-known publishers like IEEE and Springer are not alien to such obvious lack of peer review. Does it actually serve anyone's interest, that we have so many parallel "scientific" journals that publishers can't even find competent reviewers for a large number of the papers published? Does anyone follow the publications? Or are they all just a way to rip off institutional subscribers like universities, who must subscribe to all possible journals and pay outrageous sums to the publishers?
- p4bl0 13y agoThe paper you cite is a load of crap. And its author John Bohannon is a sell-out. Or at least I hope so for him, because if he compromised his intellectual faculties and his scientific reasoning for less than a very good place in hell, he is mad. This paper makes and obviously false hypothesis: that the quality of a journal depends on it being open access or not, while clearly it depends on it being seriously peer-reviewed. The real question is: among peer-reviewed journals, are those which have an open access policy worse, better, or the same as the others? (My bet is that they are the same, since scientists/researchers doing the peer reviews have no more incentive to do it better in a case or another). Bohannon shows his results in a way which can't say anything about that, mainly because he is comparing two things (open access and closed access journals) but only studied one of them and makes assumption about the other. With this level of scientific hypocrisy anyone can make numbers tell anything they want. What I take from this article is that Science is scared of what is coming. Nothing else. If they were honest they would have titled it "Who's afraid of open access?" and the content would mainly be "us".
- 001sky 13y agoThe latest discovery is merely one symptom of a “spamming war started at the heart of science” in which researchers feel pressured to rush out papers to publish as much as possible. ~Sounds about right
- Thos 13y ago"...Labbé, who has built a website where users can test whether papers have been created using SCIgen..." It's so lucky that he's built a tool to check whether a document is "computer-generated nonsense". God forbid that the publishers have to actually read the papers to check.