3 ms·

Very interesting. However what prevents this system from splitting out into virtual cliques consisting of same groups that exist today? There is no doubt that t

by abhaga 16y ago

Very interesting. However what prevents this system from splitting out into virtual cliques consisting of same groups that exist today? There is no doubt that the proposed system is more accommodating of the non-US, non-European authors. However since the RE entities are free to choose what they review, what is the incentive to dig through piles of papers from unknown authors?

lazyjeff 16y ago
According to the article, "However, authors may formally request that a particular RE review their paper. They may make only one such formal request at a time. The RE is given a short time to accept or refuse to review (e.g. 2 weeks) and a slightly longer time before the author is allowed to submit a request to another RE (whether the requested RE has produced a review or not). RE will have an incentive to review good papers, as their reputation will increase with the quality of the papers which they are the first to rate highly." I think the way this system will "bootstrap" itself is by having a few initial conferences use it as their main submission venue. Basically, this would replace a conference management system like Easychair.
- abhaga 16y ago
  Through one source or other, the system needs to have more bandwidth for processing more papers in a better way then is possible with current system. I am trying to think where does that extra bandwidth come from. As I see, there are 2 sources. One, as you pointed out from the article, is that existing reviewers put in more time and energy in the system to gain reputation. Another possibility is that since anyone can come in and start reviewing, people actually on the periphery of the system or outside the system might be able to add more bandwidth. For example, a student in India is no longer limited to review mostly for Indian conferences. He can dig around, find good papers from anywhere and give them more publicity. He can start as a feeder for more established REs and in time gain enough reputation to branch off on his own. I think this may be the major source of extra bandwidth that the system is able to harness.
tzs 16y ago
When I've sketched out designs for similar systems, I've dealt with that problem by having it acceptable to hire reviewers. Me expectation is that high karma reviewers would be in high enough demand that they could make a living as reviewers. I'd also expect that there would be science writers who would serve as reviewers to get the earliest possible access to research that might be interesting to write articles about. (I'm assuming most researchers would want to get at least some reviewer feedback before putting their paper out on some public site).

by abhaga 16y ago

lazyjeff 16y ago

According to the article, "However, authors may formally request that a particular RE review their paper. They may make only one such formal request at a time. The RE is given a short time to accept or refuse to review (e.g. 2 weeks) and a slightly longer time before the author is allowed to submit a request to another RE (whether the requested RE has produced a review or not). RE will have an incentive to review good papers, as their reputation will increase with the quality of the papers which they are the first to rate highly." I think the way this system will "bootstrap" itself is by having a few initial conferences use it as their main submission venue. Basically, this would replace a conference management system like Easychair.

abhaga 16y ago
Through one source or other, the system needs to have more bandwidth for processing more papers in a better way then is possible with current system. I am trying to think where does that extra bandwidth come from. As I see, there are 2 sources. One, as you pointed out from the article, is that existing reviewers put in more time and energy in the system to gain reputation. Another possibility is that since anyone can come in and start reviewing, people actually on the periphery of the system or outside the system might be able to add more bandwidth. For example, a student in India is no longer limited to review mostly for Indian conferences. He can dig around, find good papers from anywhere and give them more publicity. He can start as a feeder for more established REs and in time gain enough reputation to branch off on his own. I think this may be the major source of extra bandwidth that the system is able to harness.

tzs 16y ago

When I've sketched out designs for similar systems, I've dealt with that problem by having it acceptable to hire reviewers. Me expectation is that high karma reviewers would be in high enough demand that they could make a living as reviewers. I'd also expect that there would be science writers who would serve as reviewers to get the earliest possible access to research that might be interesting to write articles about. (I'm assuming most researchers would want to get at least some reviewer feedback before putting their paper out on some public site).