13 ms·
Retraction of 60 articles implicated in a peer review and citation ring
- phreeza 12y agoI know there is someone affiliated with the SAGE group who is a contributor to HN from previous comments, which I seem to be unable to find now. I wonder if they can share any details on how this was discovered, just something seeming fishy to an editor, or some kind of fraud detection algorithm?
- dougmccune 12y agoI'm on the board. I actually don't have any more information about this right now, but I'll be following up to see what I can learn. I imagine that similar to voting ring detection on a site like HN, the specific details about what tipped us off are probably going to remain semi-private. But the issue of fraud detection in academia is incredibly interesting and a good example of where publishers (both via software and human intervention) are playing an important role. I'll see if we can share any more of the technical details publicly.
- jcr 12y agoIt seems safe to assume most "known" ring detection systems being used are private/secret since their goal is to prevent abuse and revealing their details might thwart their effectiveness. None the less, at least some ring detection work is being done publicly, and I've recently been trying to learn more about the methods being used. The following should be helpful to anyone who has to fight this sort of abuse: "Doppelganger Finder: Taking Stylometry To The Underground" https://www.eecs.berkeley.edu/~sa499/papers/oakland2014-underground.pdf https://www.eecs.berkeley.edu/~sa499/papers/oakland2014-unde... Code: Find multiple accounts (doppelgangers) of a user https://github.com/sheetal57/doppelganger-finder https://github.com/sheetal57/doppelganger-finder https://psal.cs.drexel.edu/index.php/JStylo-Anonymouth https://psal.cs.drexel.edu/index.php/JStylo-Anonymouth https://github.com/psal/jstylo https://github.com/psal/jstylo
- dougabug 12y agoSounds something akin to "security through obscurity." The perceived advantages of proprietary secret security measures are often dwarfed by the benefits of opening up a system to careful scrutiny.
- pilsetnieks 12y agoIt depends. Complexity and relatively low interest in a project may mean that only a few truly motivated individuals would even look at the code - and in this case the people most motivated are those who want to abuse the system.
- danielweber 12y agoKnowing that "security through obscurity" is a bad thing is a fine heuristic, especially for beginners. Thinking "security through obscurity" is automatically and always bad is incorrect. The people running this detection software don't do it to satisfy your intellectual curiosity. They do it because they are attempting to prevent attacks on their systems.
- vardhanw 12y agoI guess it is Doug who has already responded, and the thread you are referring is this- https://news.ycombinator.com/item?id=7641398 https://news.ycombinator.com/item?id=7641398 ?
- x0x0 12y agoI would bet a lot of places that traditionally ran on trust/reputation are vulnerable to bad actors, at least once. It's sad more than anything else...
- dougmccune 12y agoJust a quick followup to this. This particular detection was triggered by a person raising a red flag that something seemed fishy with a particular email from a reviewer, which turned out to be from one of the illegitimate accounts we eventually detected. We also believe this particular fraud would have been identified regardless because we had a manual review scheduled for the most-cited articles of some of our journals, which would have unearthed this same thing. In either case, fraud detection like this seems to be a fairly manual process for us at this point. Our short-term solution to ensure this doesn't happen again for this particular journal is to throw more editors at the problem. I'm unaware of what automated systems we currently have in place or plan to have in place, but that's something I'll be following up on. If anyone happens to be working on a startup for automated fraud detection in academia I'm sure we'd love to talk :) As always, feel free to contact me directly, details in my HN profile.
- OedipusRex 12y agoWhy would someone do this?
- Blackthorn 12y agoPublication is a high-stakes game for academics.
- OedipusRex 12y agoI have heard that. If I remember correctly many colleges push for a certain number of publications per year.
- superuser2 12y agoIt's pretty difficult to lose tenure (that's the point), but publication volume and quality are pretty much the metric for hiring and promoting faculty. Just like standardized testing at the undergraduate admissions level, it's pretty much inevitable that some people will attempt to game the metric instead of what it's supposed to reflect.
- scott_s 12y agoI think that hiring faculty will tend to be primarily about publications. But promoting faculty will also depend on pulling in grants - perhaps even more so.
- logfromblammo 12y agoIt looks like there's an industry out there that would enrich a few coders and crypto experts with a tiny sum of money for a more tamper-resistant academic reputation system. It could be worth more than Bitcoin. I only wish that I already knew more about crypto so that I could both get it done correctly AND be the first solution on the market.
- p4bl0 12y agoHaving a "reputation system" at all is already problematic in the case of research. Research should not tend to satisfy arbitrary criteria. Nowadays with these h-index and impact factor crap, you see for instance journals which require (more or less officially) authors of an accepted paper to cite at least one or two other recent (1 or 2 years old max) papers from the journal. Any reputation system will work only for its own benefit and not for the benefit of research. I'm not at all saying that academics should not be evaluated. Only you can't judge the quality and pertinence of work at the state-of-the-art of so many domains with one (or even a few) number(s).
- AnimalMuppet 12y agoThere are more papers than anyone can read. So how do you tell which ones are important? And how do you objectively tell which professors are better than others at doing research? Whatever method you use, someone will try to game it.
- p4bl0 12y ago> There are more papers than anyone can read. So how do you tell which ones are important? Let the researcher decide. When an academic applies for a job he/she can attach his/her two or three most relevant papers. > And how do you objectively tell which professors are better than others at doing research? You can't, that's my point, the question makes no sense. > Whatever method you use, someone will try to game it. Then don't. Don't use a method. Evaluate a researcher by actually looking at her/his research. Yes, it takes longer to read a paper than to read a number. But the number makes no sense and who decided that we have to be in a hurry when deciding how to spend mostly-public money on research and researchers? That's actually a good question. What legitimacy have privately held journals, owned by publishers who sell back the results of research to the researchers who gave it to them for free (in the bast case scenario, sometimes you get to pay them to give it to them), to get this much influence on how to spend public money on research? Because that's pretty much what bibliometrics amounts to.
- drpgq 12y agoHow good a journal is the Journal of Vibration and Control?
- giarc 12y agoThe reputation is a bit shaky. leaves room
- new_test 12y agoJust randomly looked up one of the publications. It's not my area, so I can't talk about the meat of the paper itself, but what does the discussion section have to do with anything? http://waset.org/publications/11354/applying-half-circle-fuzzy-numbers-to-control-system-a-preliminary-study-on-development-of-intelligent-system-on-marine-environment-and-engineering http://waset.org/publications/11354/applying-half-circle-fuz...
- GrantS 12y agoIt's more of a motivation or justification of the work than anything -- they want to apply the paper's methods to control marine-related engineering systems which are becoming more important to the future of Taiwan. It sounds like a non-sequitur because it was probably copied and pasted from the grant proposal to get funding for the work. The acknowledgments show that funding came from the "National Science Council of the Republic of China, Taiwan", so that organization would naturally want to know why they're funding work on fuzzy numbers and how it benefits the country.
- new_test 12y agoRight, it just seemed like they didn't make any connection whatsoever between this particular research and a concrete naval/engineering problem. Kind of like saying "look this is math, engineering is important to Taiwan, engineers use math".
- GrantS 12y agoHa, yes, that is a pretty accurate assessment of what they did there.
- forgottenpass 12y agoI don't know anything about SAGE outside of what I've read in a psychiatrist's blog, but isn't the reputation of SAGE Publications' journals trashy already?
- GrantS 12y agoStrangely enough, I just started getting spam from SAGE this morning about subscribing to their Sociology journals. My area is computer science so it seems they're just spamming university email addresses. Not too classy.
- gwern 12y agoPapers coming out of China and Taiwan need to be regarded skeptically; this reminds me a lot of the fairly recent "China's academic scandal: call toll-free hotlines to get your name published" http://www.wired.co.uk/news/archive/2013-12/02/china-academic-scandal http://www.wired.co.uk/news/archive/2013-12/02/china-academi...
- hga 12y agoThe particular problem that article is talking about is one in the PRC, not extending to Taiwan. As I understand it, orders came from on high (by definition the Communist Party of China) driving this. Which is not to say the "publish or perish" problem isn't endemic, and that we can safely assume Taiwan is about as bad as any other "normal", non-cargo cult science country.
- gwern 12y agoBecause Taiwan is so close to the PRC, in every sense of the word, you have to worry. Has the Taiwanese government been enacting China-like policies? When researchers return to Taiwan, do they bring with them corrupt practices? Or even just beliefs in evaluating researchers by publication count? Do the revelations about Chinese papers encourage Taiwanese researchers to try their own hand at it, and normalize it? Are there any underlying cultural factors which might be the same in both countries? And so on. > that we can safely assume Taiwan is about as bad as any other "normal", non-cargo cult science country. I wouldn't say that. For example, there's a strikingly consistent observation that close to 100% of acupuncture experiments succeed when they are done in East Asia (which is pretty much impossible for any effect size) despite rates halving when you move out of the region; is the placebo effect that much stronger in East Asia, or does this reflect investigator bias / incentives to get positive results by any means etc?
- nknighthb 12y agoMy experiences with the business climates of Taiwan and China suggest to me that the set of "senses" you and I have associated with the word "close" must be vastly different.
- dang 12y agoUrl changed from http://retractionwatch.com/2014/07/08/sage-publications-busts-peer-review-and-citation-ring-60-papers-retracted/ http://retractionwatch.com/2014/07/08/sage-publications-bust..., which points to this.
- Balgair 12y agoLest we forget that all humans are created equal: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ Also, for the back and forth links on Ioannidis's 2005 paper: https://en.wikipedia.org/wiki/John_P._A._Ioannidis#Research_findings https://en.wikipedia.org/wiki/John_P._A._Ioannidis#Research_...
- templeos2 12y agoWhere can I get all the academic papers used to make the Windows operating system?
- deleted 12y ago[deleted]
- louhike 12y agoAt first, I thought it was THE Peter Chen[1]. [1] http://en.wikipedia.org/wiki/Peter_Chen http://en.wikipedia.org/wiki/Peter_Chen