4 ms·

when you review a paper, where does your "review" go? Is your review some sort of written document? Why doesn't arXiv have a place for reviews? I.e, if I trust

when you review a paper, where does your "review" go? Is your review some sort of written document? Why doesn't arXiv have a place for reviews? I.e, if I trust a reviewer and says paper is good I'm likely to go read it.

SubiculumCode 3y ago
When I write a review, I write several paragraphs to pages pointing out strengths and weaknesses, asking for clarification, new approaches, etc. I then send that to the editor who packages all the reviews together in the form of a letter to the authors. The authors in turn respond to that letter point by point in their own letter. Its more like email than a publication. In terms of whether reviews should be public, there are pros and cons to anonymous review. The cons are easy to see. When I read a paper I don't know whether the reviewers asked a question and gotten a response about a concern I have about the paper. Often clarifying results in response to a reviewer's inquiry can be shared in the letter but never make it into the paper. I always ask for a brief sentence describing those additional checks to be put into the paper briefly or in supplemental, but that is me. The pros of anonymous unpublished reviews are slightly more nuanced. Sometimes stupid but innocent mistakes happen, and these can be embarrassing. Part of the value of review is having the chance to have fresh eyes on it, but doing so in public view can push people to get grounded in their position as a matter of reputation. Another issue is that criticizing a paper from a well known research is a bit harder when it will be published, and especially when it isn't anonymous.
pdonis 3y ago
It does: other papers written in response. I think the real question is, can arxiv show me other papers that were written in response to this one?
derbOac 3y ago
That's a good question. Some of these academic preprint/archive sites are experimenting with "a place for reviews". There's a couple of issues (I hesitate to say "problem" but it could be seen that way): 1. How do you choose who does reviews? 2. Why should the archive service host those reviews? Currently, in general, an editor asks reviewers to review, they review, and it's part of the editorial process leading to publication. This is a sort of "gatekeeping" model, for better or worse. If you let reviews into an archive service, how do you do that? If you let everyone do it, that's fine, but then do you need arXiv etc to host those reviews? Should arXiv etc host those reviews? If you do have reviewers selected, how do you do that? I'm not saying there aren't answers to these questions, but it does get a little complicated. Pay-to-publish has its own huge problems imho for example. In general, academics don't know what to do to mark value in the absence of a "stamp of approval" from an "endorsed expert". I think that's what's underlying all of this.

SubiculumCode 3y ago

When I write a review, I write several paragraphs to pages pointing out strengths and weaknesses, asking for clarification, new approaches, etc. I then send that to the editor who packages all the reviews together in the form of a letter to the authors. The authors in turn respond to that letter point by point in their own letter. Its more like email than a publication. In terms of whether reviews should be public, there are pros and cons to anonymous review. The cons are easy to see. When I read a paper I don't know whether the reviewers asked a question and gotten a response about a concern I have about the paper. Often clarifying results in response to a reviewer's inquiry can be shared in the letter but never make it into the paper. I always ask for a brief sentence describing those additional checks to be put into the paper briefly or in supplemental, but that is me. The pros of anonymous unpublished reviews are slightly more nuanced. Sometimes stupid but innocent mistakes happen, and these can be embarrassing. Part of the value of review is having the chance to have fresh eyes on it, but doing so in public view can push people to get grounded in their position as a matter of reputation. Another issue is that criticizing a paper from a well known research is a bit harder when it will be published, and especially when it isn't anonymous.

pdonis 3y ago

It does: other papers written in response. I think the real question is, can arxiv show me other papers that were written in response to this one?

derbOac 3y ago

That's a good question. Some of these academic preprint/archive sites are experimenting with "a place for reviews". There's a couple of issues (I hesitate to say "problem" but it could be seen that way): 1. How do you choose who does reviews? 2. Why should the archive service host those reviews? Currently, in general, an editor asks reviewers to review, they review, and it's part of the editorial process leading to publication. This is a sort of "gatekeeping" model, for better or worse. If you let reviews into an archive service, how do you do that? If you let everyone do it, that's fine, but then do you need arXiv etc to host those reviews? Should arXiv etc host those reviews? If you do have reviewers selected, how do you do that? I'm not saying there aren't answers to these questions, but it does get a little complicated. Pay-to-publish has its own huge problems imho for example. In general, academics don't know what to do to mark value in the absence of a "stamp of approval" from an "endorsed expert". I think that's what's underlying all of this.