4 ms·
As a former researcher with multiple high-impact publications I can 100% confirm it: - when I was a researcher in Italy for an unknown lab, all of our articles
by epolanski 4y ago
As a former researcher with multiple high-impact publications I can 100% confirm it:
- when I was a researcher in Italy for an unknown lab, all of our articles were generally very scrutinized and went through years of reviews before publication
- when I worked in Michal Graetzel[1]'s laboratory things get published much more easily on more higher impact journals with less scrutiny. Not only most of the publications didn't really add much to the scientific knowledge, but they didn't even have the very high standards required when I was in Italy
Why is this bad? You get more funding the more you publish...So labs that publish easier due to the names involved get much more money...which they can use to publish even more and get even more money...which they can use to publish even more...
There's excellent scientists anywhere, really. But funding, fame and politics are extremely asymmetric in academia.
[1]https://en.wikipedia.org/wiki/Michael_Gr%C3%A4tzel https://en.wikipedia.org/wiki/Michael_Gr%C3%A4tzel
- jonathanstrange 4y agoI don't understand this. Don't you have blind peer review? In my discipline, most journals even have double blind peer review. I thought that's the standard.
- triknomeister 4y agoMathematics doesn't have double blind peer review unfortunately. It will help in a big number of cases, even if some cases fall off (for example, if you do experiments on a well-known software which only one lab works on, then double blind will have a limited impact.)
- KvanteKat 4y agoAs the linked article points out, practices can vary widely across fields (and in fields where preprints are available, maintaining authour anonymity in a double-blind setting furthermore relies on revieweres not having come across a preprint of the paper that is being reviewed which in turn can easily happen in smaller research-communities).
- epolanski 4y ago> Don't you have blind peer review? It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics and conferences make those circles even more public. Say that you are studying e.g. "helical peptides", which is a vague and broad term on one side, on the other one, all the people that work on specific aspects of helical peptides (say, using them to bind surfaces that are very different in polarization, e.g. batteries) know each other and blind review would add nothing really. Even when it is blind you likely know who's reviewing it most of the time. There are topics that are of course wider. To make a comparison with programming: imagine you're involved in an open source community with a language that is niche enough that you know most of the people involved in it (it is common for many lisp dialects). You can likely be given a program in that language and figure out likely who wrote it due to their interests, programming or api designing, or the use of static types, or some kind of linting that you know only a handful of people use, background, etc. With the huge difference that in case of science the circles are much smaller because you know all your peers are in academia, who of them has specific instruments (not everyone has the same lab equipment), but who doesn't also have some others, you may know that some of your peers have their niche and "main branches" of research so you figure those things much easier. Also, in those circles you also know who is involved in your "code reviews" so if you know that A and B aren't reviewing it (you know they would've told you) it can only be C, D and E, but the feedback requesting this kind of experiment could only come from "C"...
- techas 4y ago>It's fundamentally impossible because when you publish the topics are going to be in a way or another niche enough that you know all of the people working on those topics Even if you know the people in the field and you can guess the authors, double blind does not hurt. Actually, I don’t see any downside of double blind. Actually, names are very often used to let you out of the field if you are not part of the usual family…
- Orothrim 4y agoYou can't see any downside of double blind? There are many systems we use which provide anonymity, and they always result in some form of abuse. Such as phone numbers (scam calls), Internet (cyber harassment, scams, viruses), cryptocurrency (all the scams). Any system where humans can gain advantages from abusing anonymity they do so. I'm not an academic so maybe there are already safe guards, but from my understanding people already cut any research into as many small papers as they can. Truly double blind peer review would likely encourage this. Maybe it would also further encourage people to steal research or peer review outside of their expertise. The benefits may outweigh the negatives, but I'm sure people will find a way to abuse it too, consciously or not.
- rusticpenn 4y agoYou can easily find the authors based on the references they use and the topic.
- jonathanstrange 4y agoSure but the idea is that you don't try to find out who the author is, and if you recognize the author anyway, you decline the review. I review regularly and that's what I do whenever I recognize an author.
- pbhjpbhj 4y agoDon't you recognise authors on topics you're most competent to review on?
- jonathanstrange 4y agoNot very often in a way that causes me to decline to review. What happens sometimes is that you have a suspicion but it's not definite. It could be one out of a dozen authors, or a newcomer you've never heard of. I'm in the humanities, however, things probably look different in the natural sciences.
- rusticpenn 4y agoNot everyone seems to have your ethics.
- kalimanzaro 4y agoWhat is your discipline? I'm in physics and double blind peer review is not the norm.
- caddemon 4y agoWell in various CS fields people preprint on arXiv these days, so it is easy for reviewers to unblind themselves. Sometimes accidentally, because preprint timing and subsequent Twitter advertising coincide so well with conference deadlines...
- lupire 4y agoWhy doesn't arxiv allow name-blind preprint posting?
- caddemon 4y agoThat would be a good idea, and then have the names revealed in a future version like the way the paper itself can be updated. I don't think it is possible though. If it is, it certainly isn't used.
- ljw1001 4y agoWithout peer review, publications without names would be untrusted and untrustworthy, and would probably represent a security issue
- caddemon 4y agoThe main point of arXiv is not to substitute for actual publication. Some people use it that way, which could still occur. But most intend to submit their work for review to a venue with a supposed double blind process. The use of arXiv is for flag planting and ease of open access. Often the arXiv upload will be right around the time of a conference deadline in fact. For these people I see no problem with an OpenReview-esque setup where the arXiv page remains anonymous until the authors opt to reveal themselves. If people don't care for true double blind review then they should stop pretending they do. Conferences like ICLR and NeurIPS are far from blind. It would be one thing if it were just a matter of trusting reviewers to not Google the paper title, but it goes beyond that. There are currently no restrictions on the use of social media to advertise works under review nor on the timing of arXiv uploads, which can trigger alerts to those in the field right around review time.
- avidphantasm 4y agoEven with blind reviews, many people cite their prior work extensively, so it can be hard to truly obscure who the paper authors are.
- jcfrei 4y agoHeard from colleagues working in academia that their profs were absolutely strict about the layout, the way graphs look, the colors which needed to be used, etc. All so reviewers could decode who's actually behind a paper.
- dataflow 4y agoNot every conference or journal is blinded, no.
- asah 4y ago"double blind" is not close to achieved, given differences in writing style, paper structure, graphics, choice of citations, and and and...
- kalimanzaro 4y agoI suspect the mechanism goes something like this: Reviewers prioritize their time above all, and after that correctness,leaving novelty in last place. If a paper comes from a unknown lab it tends to get greater scrutiny because a famous name is a subconscious stand-in for correctness. You spend less time reviewing famous authors because you think they are less likely to have bugs.
- lupire 4y agoI think "aligned with reviewer's biases" is more important than correctness or novelty.
- mattkrause 4y agoReviewing a paper is effectively unrewarded: you need some kind of reviewing service on your CV, but the amount and quality is barely measured, let alone considered for career progression. Reviewing a very good paper is quick and easy ("LGTM"). Really terrible papers are not too bad if they're obviously awful, but reviewing something subtly flawed takes a lot of work: you need to identify the flaws and describe them in a way that's compelling enough to convince the authors--or at least the editors. Ideally, you'll also explain how to address them, which is more work now and down the road when you review the authors' often-grudging implementation of your suggestion. In such a world, people may be increasingly reluctant to review papers without some indication of their quality (for which name is a rough proxy). My solution to this is to somehow make reviewing more valued. It's an important part of science and deserves more than a checkbox.
- glofish 4y agoI think this is the best point here. Reviewing is hard work with very few rewards - and you have to decide at the start whether it is worth your time to embark on it. Ideally we would like to learn something from the paper. I and most people here would most likely strongly prefer to review a paper written by someone good at what they do. In reality most papers are pretty bad - thus when we prefer reviewing papers written by Nobel prize winners we are not "biased" and "humbly bowing" before their greatness - we are just trying to make it worth our while.