3 ms·
The quality of reviews of conference papers is mostly terrible, though. I've been on both sides of it, here are some major complaints: * There is only one roun
by woodson 10y ago
The quality of reviews of conference papers is mostly terrible, though. I've been on both sides of it, here are some major complaints:
* There is only one round of reviews, with a binary accept/reject outcome. In case of rejection, feedback cannot be incorporated. In case of acceptance, there is basically no incentive for incorporating feedback, as the paper is already "through" and putting in any more time is considered a waste.
* Assigning papers to reviewers is very arbitrary and often the reviewers are not at all familiar with the field. Getting a paper reassigned to another reviewer is often so complicated and time-consuming that reviewers just do the bare minimum (if asked to rate novelty/originality etc., on a scale 1-5 they give it a "3", which given the usual acceptance rate of <50% means the paper will get rejected).
* Reviewers usually get a lot of papers to review. They don't have time (who does?) but have to finish by a two-week deadline. If one of the papers is relevant to the research of the reviewer or has been written by colleagues they know (double-blind does not help, you recognize who wrote it..), that paper will receive attention; the other papers are seen as distraction and are dealt with as outlined above.
Peer-review is important, though. Some fields of research, particularly those where a lot of research is done by actual practitioners and not just pure research scientists, are mostly driven by conferences with short (one page or less) abstracts without full papers. There are often very few citeable sources that have been peer reviewed; conference websites and abstract books vanish over the years. An arxiv-like repository can help with that, but it does not solve the problem of peer review. Community-run open-access journals seem to provide a solution, but rarely take off.
- cLeEOGPw 10y agoSo that sounds like the review system is broken, but not the concept itself. I always had the impression that anyone could review the paper, as in anyone can read a book and write a review about it, and then they all publicly could ask questions about parts of paper that are incorrect, science paper "bugs" if you will, that the author would have to clarify/fix and when there are no more of these "bugs" it is considered as peer reviewed. Although it wouldn't mean there are no more incorrect things, just that nobody asked about them. Also that would mean that mostly people interested in field are involved in review. What you described sounds really detrimental to the whole process. Is peer reviewing so hated between researchers that they need to be assigned and forced a deadline to get papers reviewed?