3 ms·

Imagine that you have voting rights to review a paper (a-la slashdot, where random people got opportunity to tag something as insightful, interesting, etc). No

Imagine that you have voting rights to review a paper (a-la slashdot, where random people got opportunity to tag something as insightful, interesting, etc). Now imagine that there comes an article in X subject (say, Agent Based Modeling). When you "vote" in that article, the "dimension" of your vote is proportional to your "impact factor" in that subject (i.e., say you published 20 articles in ABM and you got 10 "votes" on them, then each of your votes count as 10 votes). On the contrary, if your impact factor is negative, your vote doesn't count. That way people that are considered "knowledgable" in their subject, will be able to peer-review other articles. Another method would be something like what StackOverflow has: Initially everybody gets 1 vote (or 10, or 1 every month, or whatever), and you "transfer it" by voting for an article (maybe to the 1st author, or evenly distributed), so because the "votes" are scarce, people with care for them. And people with articles that are most voted, can themselves vote more. There are plenty of systems that could work. And the beauty of it is that they could be "layered" on top of Arxiv with a Chrome extension or similar. Mhmm, sounds like a good weekend project .

jlokier 6y ago
I think what you are describing has a name: Reputation Systems (https://en.wikipedia.org/wiki/Reputation_system https://en.wikipedia.org/wiki/Reputation_system). There seem to be more effective ones and less effective ones. Unfortunately sometimes economic incentives cause them to be gamed against real quality. E.g. think of shill online shopping reviews, and circular voting to boost reputation. In academic terms, that would be "citation rings" to promote their rankings. Like web rings, there are nice and friendly ones, and there are heavy, spammy clones of sites. I would expect the rise of junk-article, plagiarism-from-elsewhere, machine-learning-assisted-plausibility mutual citation rings if there were no good controls to detect and prevent that sort of thing.

jlokier 6y ago

I think what you are describing has a name: Reputation Systems (https://en.wikipedia.org/wiki/Reputation_system https://en.wikipedia.org/wiki/Reputation_system). There seem to be more effective ones and less effective ones. Unfortunately sometimes economic incentives cause them to be gamed against real quality. E.g. think of shill online shopping reviews, and circular voting to boost reputation. In academic terms, that would be "citation rings" to promote their rankings. Like web rings, there are nice and friendly ones, and there are heavy, spammy clones of sites. I would expect the rise of junk-article, plagiarism-from-elsewhere, machine-learning-assisted-plausibility mutual citation rings if there were no good controls to detect and prevent that sort of thing.