4 ms·

Readers commenting on papers would be a welcome addition. Hopefully, with a reputation system to go with it.

by j2kun 10y ago

deleted 10y ago
[deleted]
argonaut 10y ago
I'm not an academic, but I think you'll find many academics would be against this. Things would get more complicated because there would obviously need to be a vetting process on who could comment. Things shouldn't be a popularity contest, so I think a reputation system would be counterproductive.
- noobermin 10y ago
  >Things shouldn't be a popularity contest You clearly are not an academic. Publishing papers is a popularity contest.
  - justicezyx 10y ago
    But it's a different type of popularity contest. Commenting on publications is virtually non-existent. It's not that there is no venue of commenting, we do have plenty of academic conferences; but the amount of comments received in the conferences is virtually none. People do ask questions, but seldom give constructive (be it positive or negative) comments.
    - adpouasd 10y ago
      I don't think this is true in general (though it might be so in the field you are working in). I've received many constructive comments from colleagues, and this has often lead to subsequent collaboration.
    - GFK_of_xmaspast 10y ago
      Pubpeer?
  - argonaut 10y ago
    Are you an academic? Because peer review is almost always (at least in CS) blind, with authors anonymous to the reviewers.
    - santaclaus 10y ago
      Well, I find CS venues (conferences and journals) to be about 50/50 in terms of blind or not. Also, while it will certainly vary subfield to subfield, once you are familiar enough with an area you can pretty confidently guess what group (if not what exact authors) are behind any given paper. I bet it would be pretty easy to cook up a automated paper un-anomymizer based on the body of published work out there already.
      - noobermin 10y ago
        In my field of laser physics, there are some really, well known lasers, so if your paper talks about this or that facility and cites this "monumental work by X group" you can easily gather who is the author. Not to mention (yes heaarsay) of at least one person in my group who refereed a paper, harshly criticizing it paper only to have the editor come down and say "accept this paper anyway." Academia has some good parts but it's as political as anything, if not more so.
paulsutter 10y ago
The fact that reader comments are controversial (about as many people are opposed to this as in favor, see graph in article) suggests that it's a great idea, and that the design of the reputation system is critical. Perhaps they can semi-hide the comments during an introductory phase where they develop the reputation system. A well-crafted reputation system could eventually take on a role resembling peer review, but should be approached very carefully.
smaddox 10y ago
Reputation systems are nice for providing a carnal motivation, but moderation is likely going to be necessary to maintain a high quality density. Crowd-sourced moderation a la slashdot is a reasonable place to start. I think there's a lot of room for improvement in this space, though. An academic publishing I hope that in the future, something like ArXiv with a discussion system will be built on a fully distributed network a la IPFS. That's been one of my most ambitious dream projects since my third year of grad school. I haven't made any progress towards it. I think such a system could stave off some of the decline in academia. I'm glad that ArXiv is headed in that direction. They're the experts.
santaclaus 10y ago
I would love to have some kind of stack overflow type system for commenting on papers. If you find a bug in some paper chances are someone else will benefit from the knowledge. Oh, the authors didn't report some key step in the algorithm and you figure it out? Pay it forward and leave a comment! Crowd sourced errata, if you will. Hell, integrate with GitHub so you can link to implementations, etc.
- fuzzythinker 10y ago
  http://gitxiv.com/ http://gitxiv.com/
luaskldj 10y ago
I don't think this is a useful thing to have. There is no clear need for it, and it would likely reinforce group think a la HN or reddit.
deleted 10y ago
[deleted]

Readers commenting on papers would be a welcome addition. Hopefully, with a reputation system to go with it.

by j2kun 10y ago

Readers commenting on papers would be a welcome addition. Hopefully, with a reputation system to go with it.

deleted 10y ago

[deleted]

argonaut 10y ago

I'm not an academic, but I think you'll find many academics would be against this. Things would get more complicated because there would obviously need to be a vetting process on who could comment. Things shouldn't be a popularity contest, so I think a reputation system would be counterproductive.

noobermin 10y ago
>Things shouldn't be a popularity contest You clearly are not an academic. Publishing papers is a popularity contest.
- justicezyx 10y ago
  But it's a different type of popularity contest. Commenting on publications is virtually non-existent. It's not that there is no venue of commenting, we do have plenty of academic conferences; but the amount of comments received in the conferences is virtually none. People do ask questions, but seldom give constructive (be it positive or negative) comments.
  - adpouasd 10y ago
    I don't think this is true in general (though it might be so in the field you are working in). I've received many constructive comments from colleagues, and this has often lead to subsequent collaboration.
  - GFK_of_xmaspast 10y ago
    Pubpeer?
- argonaut 10y ago
  Are you an academic? Because peer review is almost always (at least in CS) blind, with authors anonymous to the reviewers.
  - santaclaus 10y ago
    Well, I find CS venues (conferences and journals) to be about 50/50 in terms of blind or not. Also, while it will certainly vary subfield to subfield, once you are familiar enough with an area you can pretty confidently guess what group (if not what exact authors) are behind any given paper. I bet it would be pretty easy to cook up a automated paper un-anomymizer based on the body of published work out there already.
    - noobermin 10y ago
      In my field of laser physics, there are some really, well known lasers, so if your paper talks about this or that facility and cites this "monumental work by X group" you can easily gather who is the author. Not to mention (yes heaarsay) of at least one person in my group who refereed a paper, harshly criticizing it paper only to have the editor come down and say "accept this paper anyway." Academia has some good parts but it's as political as anything, if not more so.

paulsutter 10y ago

The fact that reader comments are controversial (about as many people are opposed to this as in favor, see graph in article) suggests that it's a great idea, and that the design of the reputation system is critical. Perhaps they can semi-hide the comments during an introductory phase where they develop the reputation system. A well-crafted reputation system could eventually take on a role resembling peer review, but should be approached very carefully.

smaddox 10y ago

Reputation systems are nice for providing a carnal motivation, but moderation is likely going to be necessary to maintain a high quality density. Crowd-sourced moderation a la slashdot is a reasonable place to start. I think there's a lot of room for improvement in this space, though. An academic publishing I hope that in the future, something like ArXiv with a discussion system will be built on a fully distributed network a la IPFS. That's been one of my most ambitious dream projects since my third year of grad school. I haven't made any progress towards it. I think such a system could stave off some of the decline in academia. I'm glad that ArXiv is headed in that direction. They're the experts.

santaclaus 10y ago

I would love to have some kind of stack overflow type system for commenting on papers. If you find a bug in some paper chances are someone else will benefit from the knowledge. Oh, the authors didn't report some key step in the algorithm and you figure it out? Pay it forward and leave a comment! Crowd sourced errata, if you will. Hell, integrate with GitHub so you can link to implementations, etc.

fuzzythinker 10y ago
http://gitxiv.com/ http://gitxiv.com/

luaskldj 10y ago

I don't think this is a useful thing to have. There is no clear need for it, and it would likely reinforce group think a la HN or reddit.

deleted 10y ago

[deleted]