8 ms·
Elsevier investigates hundreds of peer reviewers for manipulating citations
- xhkkffbf 7y agoThis seems kind of bogus to me. Yeah, I can see how someone could construe it as such, but I think there's a perfectly legitimate reason to mention possible citations in the review. If I get a paper that covers something I've written about in the past, I feel perfectly justified saying, "Gosh, this has been published before. Perhaps you should cite this and explain how your work is new." I do it for other papers and I don't see why it would be wrong to do it for my own. Oh, I can understand how someone can twist it sound nefarious, but that doesn't seem the best solution to me. More citations to prior work is a good thing. More discussion about what is truly novel is a good thing. And let's be honest, my salary won't change one iota if I get another few citations. The idea that this is somehow selfish is really bogus. Citations aren't worth that much at all.
- bonoboTP 7y agoThat's reasonable. However in many cases reviewers push some very tangentially related paper of theirs that is far from what you describe.
- lutorm 7y agoThis is true. But did you actually read what the study said? They point out that there is a small number of referees who consistently have their own work cited at a much higher rate than the majority of the referees. If referees and authors are semi-randomly matched, there's no reason a small number of referees should consistently get to review papers that have incorrectly omitted references to their own papers.
- namirez 7y ago>And let's be honest, my salary won't change one iota if I get another few citations. The idea that this is somehow selfish is really bogus. Citations aren't worth that much at all. I'm not sure if this is true. Citations are the primary currency in academia. It's not about money; it's about prestige. >More citations to prior work is a good thing. More discussion about what is truly novel is a good thing. Again I'm not so sure about this. The problem is that most papers are not novel at all. There are some seminal papers in each subject that everyone cites. There are also papers that are closely related to what someone is doing. Everything else is usually a distraction.
- diffeomorphism 7y ago> And let's be honest, my salary won't change one iota if I get another few citations. The idea that this is somehow selfish is really bogus. Citations aren't worth that much at all. IF you already have a permanent position and many papers. While you are still a young researcher with fewer publications, hiring committees definitely are also interested in whether your papers are highly cited (see also the whole notion of "impact factor" of a journal).
- itcrowd 7y agoYes, this is correct. Additionally, massive cash incentives are being handed out in (at least) China [1]. This reward is given based on the impact factor of the journal, which means that excessive citation of your papers can artificially inflate the IF of the journal and lead to higher payout for your next article(s). It's a bit of a stretch, but it could have an impact if it is done on a massive scale. [1] https://www.technologyreview.com/s/608266/the-truth-about-chinas-cash-for-publication-policy/ https://www.technologyreview.com/s/608266/the-truth-about-ch...
- nathanaldensr 7y agoEveryone is being told to put faith into "science" as somehow incorruptible, only to have articles like this written. We are supposed to be able to rely on the people writing papers being ethical, when it seems the reality is a race to the bottom like everywhere else in society.
- Thriptic 7y agoWell, the problem in this case isn't really in the scientific or peer review process itself, it's in using citation count as a metric for paper quality or researcher expertise. I've had people play this game on several of my papers under review. Reviews will say something like "high quality submission but the authors should consider including work X for completeness" etc where it's obvious that work X was written by the reviewer. As an author, I just found some place to plausibly cite the work and moved on. The quality or validity of my manuscript was not impacted in any way. In my admittedly limited experience, reviewers will rarely green light obviously inferior work just to get an additional citation as it impacts their reputation and it's an inefficient way to stat pad (self-citation or trading citations with your collaborators is by far the most efficient way to pump citation count, especially in the context of review papers). Additionally, obviously inferior works don't even make it to the peer review process; they are rejected by the journal editors in a first pass screen. There are more insidious games people play that DO impact publication quality, such as requesting friendly reviewers (this mechanism is normally legitimately used to help editors route manuscripts for review to domain experts), submitting work to journals where the PI is an editor / has relationships with editors, attaching well known PIs to manuscripts to add the appearance of legitimacy even though they did no work and were not involved at all, etc. These are problems that won't go away because there are far too many researchers and not enough money / positions to go around, so people are heavily incentivized to play games.
- kerkeslager 7y agoScience is incorruptible. Scientists are not. Don't conflate the imperfections of the humans implementing science as being problems with science. Whatever you're proposing as the alternative to science has the same problem: it's implemented by imperfect humans. What's your alternative?
- buboard 7y ago... and then they 'll be after networks of ~~bots~~ reviewers , and then networks of networks of bots, and then state actors etc etc. Scientific publishing quickly catching up to twitter and blackhat SEO
- neilv 7y agoThis seems a positive move for science process, as well as for Elsevier's reputation and the value it can add as a publication venue.
- semi-extrinsic 7y agoI think undoubtedly the reason for "investigating" this now is the optics for Elsevier. They are bad guys in the eyes of most people right now. The main evidence I have for this claim, is that coercive citations are so common as to be a trope in PhD comics and the like. Moreover it is bleeping obvious to the editor, when they receive a review by M. Ister that suggests 10+ new references to M. Ister et al., that this is going on.
- bonoboTP 7y agoIt also de facto deanonymizes the reviewer.
- ganitarashid 7y agoThat may be the least of their problems right now
- einpoklum 7y agoI don't like Elsevier, but I always wondered: Why aren't there a few meta-reviewers, to review the reviews? That is, why doesn't the PC chair, or some key reviewers, have a look at the reviews given on papers to make sure that: - They were respectful - They were to the point - They did not indicate obvious lack of knowledge of the relevant subject matter (this one might be a bit tricky) - They were not self-promoting That would really have helped a lot of conferences... (but I actually know the answer: There's a dearth of reviewers as-is, so there's de-motivation to introduce this meta-review mechanism even if it would help review quality.)
- Majromax 7y ago> Why aren't there a few meta-reviewers, to review the reviews? In principle, the associate editor managing the submission (and making the ultimate accept/reject decision) should be doing this for every paper. In practice, the workload is such that editors have an incentive to make the review process as automatic as possible on their end.
- einpoklum 7y agoSo in theory, a highly-trusted reviewer should avoid regular reviews and be engaged in meta-reviews, because of that load.
- lhorie 7y agoIf the problem relates to the integrity of reviewers, I suspect you can't fundamentally fix the problem by adding more reviewers.
- einpoklum 7y agoSure you can: People examining each other's work deter corruption and also have a good chance of noticing it. Only if a reviewer and a meta-reviewer collude to mis-review do you have a problem, and that should be rather rare.
- 7y ago
- einpoklum 7y agoUnrelated to anything else: Please try avoiding work with Elsevier journals, and instead prefer open-access not-for-profit ones. Did you know Elsevier had a 37% operating profit margin last year?
- denton-scratch 7y agoElsevier has an open-access no-cost publishing option [1]. I understand it's considered inferior to the paywalled version by reviewers, and that Elsevier itself regards it rather like a loss-leader. So indirectly, I suppose Elsevier profits from it. And I don't disagree with the exhortation to avoid Elsevier - it's better for all of us if research (and especially publicly-funded research) can be read by anyone, without having to pay a gatekeeper. [1] https://www.elsevier.com/en-gb/about/open-science/open-access https://www.elsevier.com/en-gb/about/open-science/open-acces...
- itcrowd 7y agoHere's a possible solution to this problem. I'm still workshopping the idea, so feedback is welcome. The review process must become more transparent. The reviewers and their assessment of the work must become public when the paper is accepted and published. This in itself will prevent some nefarious citation-begging. In addition, some guidelines could be mandated concerning citations to be added or removed. For example, for each reference you want the author to include, you must explain why it should be included. Not just "this has been done before" but "the work you describe is similar to the work done on p.4 of [new ref.] but differs in X,Y and Z and is still novel because of invention/method A,B,C. Therefore, you build on their work and should cite them." To the others in this thread: remember that peer review is often an iterative process and the authors and reviewers have multiple rounds of review in which the authors can choose not to implement some suggested changes, such as adding citations. This is often a small debate. The editor of the journal has the final say and can overrule the decisions of the reviewers (publish / reject) if there are good grounds to so. Also a small note: the reviewers are selected by the journal editors based on expertise. That means, especially for smaller fields, that many prominent papers are written by the reviewers simply because they are the experts. It is easy to conclude that this leads to bad behavior but it is often the case that there simply are no other papers available with such specific details. Of course, this is not an excuse for malicious actors which are also rampant.
- kayaeb 7y agoNot sure why you're being downvoted I disagree on one point specifically though. Reviewers must remain anonymous. What if I harshly review / reject a crap work from the department head of some group at Harvard? then, in anger, I get blackballed by s/he and their friends, which could be the whole field in some cases?
- diffeomorphism 7y agoThe reviewer is known to the editor, so you already have some accountability there. The reviewer should definitely NOT be known publicly for the same reason that reference letters are usually confidential. You are going to get sugarcoated, bland, self-censored reviews otherwise. > "the work you describe is similar to the work done on p.4 of [new ref.] but differs in X,Y and Z and is still novel because of invention/method A,B,C. Therefore, you build on their work and should cite them." That is your job! As a referee I will point out that: - This is similar to paper X, which you did not cite. You should give credit here. - You, not me, should argue why your work is new/better etc. and hence worth publishing. If you just write "we do the exact same stuff as in X", then I will assume this is not novel and not that you are lying and it is actually novel and very different from X.
- jshowa3 7y agoElsevier has always smelled bad to me. They published the Andrew Wakefield autism paper for crying out loud.
- denton-scratch 7y agoWell, The Lancet published it - The Lancet is probably the #1 medical scholarly journal bar none. True - The Lancet is owned by Elsevier. Ten years later, the General Medical Council found that Wakefield's research had been dishonest, and on that basis The Lancet retracted the article. But I don't think it's right to criticise Elsevier over the publication of a paper that wasn't conclusively falsified for another decade. Reviewers are not supposed to be detectives; it's not their job to expose deliberate fraud. They rely on the material submitted by the author. This is the much same as for auditors; they rely on the documents supplied by the auditee's accounting team, they're not in the business of investigating criminality and blatant lying. Elsevier smells no worse to me than any other commercial publisher of scholarly journals. To be clear, I think they all have a pretty dodgy whiff about them; but I'm not aware that Elsevier is any worse than the others.
- jshowa3 7y agoMost papers I've heard of that exhibit malfeasance come from Elsevier and it's my understanding that the journal editors manage the whole peer review process and are compensated by the publisher through some form of an agreement. And yes, it is the job of reviewers to at least recommend not publishing a paper that can be detected as fraudulent.
- kayaeb 7y agoI'm actually surprised the number is so low. With high profile scientists, in specific fields there are often 2 camps pushing 2 different competing solutions to the same problem, and each solution tree has a set of "canonical" references which are cited for most papers in that tree. A set of these is generally requested to be discussed by the opposing team (i.e. why does your paper support your tree and what have you found that seemingly invalidates the other tree?) The more prolific authors in a subfield are generally chosen as reviewers, and also hold the most recent papers in that tree, so requesting citations to recent relevant literature often goes hand in hand with work they've had input on. Add this to the fact that, even in blind reviews, the reviewer can guess the author's name correctly more than half the time, and the reverse is anecdotally true (this is not as astounding as it seems, in my specific subfield there are maybe 20 people putting out a paper a year -- there was a paper I reviewed where I did not suggest my own work for citation, but the author retroactively added a citation for me, leading me to believe they guessed my identity), and I'm surprised less than 1% "consistently" are cited in papers they review. If anything, this meta-analysis is heartening, as I would expect this problem to be much more prolific in the "publish-or-perish" (get cited or lose your job) paradigm. Elsevier, in my field, has high impact journals though, and I know the problem to be much more rampant in the lower level journals. As a slight aside, some countries simultaneously promote local researchers to publish in their own local journals and turn around to note that publications by their scientists have tremendous citation numbers, I personally question the validity of those citation numbers since they seem to be politically incentivized. Within the fields, real researchers will have a feel for what journals are pushing good science and which are "citation padding" with incomplete and hasty work just "published" for numerics. In response to the meta-review comments. All review is volunteer work, it should be this way, but I personally find it obnoxious when I'm on public wifi and can't even load a paper I've reviewed because some ostentatiously profitable publisher has paywalled it. The research is publicly funded, the researcher pays 100$ per page of publishing cost, the review is volunteer work. Why exactly do you need 30$ for me to read the methods section?
- evandijk70 7y agoI agree with most of your comment, but why do you believe review should be volunteer work? In lots of non-academic fields you have professionals evaluating the quality of products, and being paid to do so.
- timothychan 7y agoWould be interesting to see how this unfolds, and how blockchain can come into play in this area.
- throwawayjava 7y ago"grifters complain about free labor"
- eyegor 7y agoIt's good that they're investigating the "insider dealing" varient of citation boosting but I haven't seen anything around excess self or chain-citing, which I always viewed as a more common offense. Obviously follow-up papers are a good thing, but I've known several authors that aim for long chains of follow-up papers (or summarizing several chains) purely for boosting citation numbers. The idea being that if it's a hot topic their paper will be caught in the crossfire regardless of whether it truly merits a citation.