5 ms·

I was assuming that the article was suggesting a system that is similar to reddit, but with important differences. For instance, requiring real names and publi

I was assuming that the article was suggesting a system that is similar to reddit, but with important differences. For instance, requiring real names and public voting records. Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. There are probably other things you could do to.

radarsat1 15y ago
> Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. I see an immediate problem there. There are magnitudes more people _outside_ any given area of expertise than inside. Even small votes add up.
- juiceandjuice 15y ago
  Ladies and gentleman, I present to you the logarithmic scale.
rlpb 15y ago
Apart from the lack of anonymity, I think that the ideals of such a system are very similar to the ideas of any social voting system (such as Stack Overflow, Slashdot, reddit, HN). The goal is to simply have a rating system that works, right? The trouble is that we still haven't managed to do this well in the technology world. And some of us believe that we need to go back to some level of curation. I'd love to see a system as described that actually works. But until we (the tech world) manage to figure out a system that works, I'm not convinced that scientific publishing will fare any differently.
- Joakal 15y ago
  The technology world already has it; We have Wikipedia (Edit history), Advogato (Paths), StackOverflow (Rep), etc that encourage personal profiles. You are more valued as a loyal member than one that just started. Unlike Reddit, Slashdot and HN that do not encourage personal profiles. Do you even remember my name? What about other HNers besides pg? Why do you read their posts anyway? They work because it's a form of almost pure democracy being applied and have the same problems of it eg tyranny of the majority. What the scientific model can do is a multitude of above; institution API confirming credentials (Referral), history (paths, did this person do other reviews?), etc. Cheap and still effective peer-review. I'm not sure which kind of government type applies to my suggestion above, but there are many forms of government they can apply. More: https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms_of_government https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms...
rflrob 15y ago
Even cooler would be to have each person (explicitly or implicitly) set other people's vote importance. I can imagine a complicated system wherein voting cliques are detected, and used to feed back into the rankings of what papers show up on your page. Ideally, the voting could be more than just "+1/-1" as well, potentially with a couple different axes, so that I could, for instance, say that I think a paper is important to the field, but likely wrong. Just because a paper is controversial doesn't mean I don't want it to be discussed (for instance, if it introduces a new computational technique, but the input data is flawed somehow).
- wickedchicken 15y ago
  You're matching an engineering solution to a social problem. Trying to map a paper's social significance to an abstract number is really hard, and in the process you lose a lot of ability to present papers based on taste. Showing me a paper because "these numbers are high and you tend to like these kinds of numbers" is fundamentally different than "you should see this because it is groundbreaking." On the other hand, if we can create a metric for elegance and another for "brings the reader closer to enlightenment," then I'm all for it.
- chalst 15y ago
  The property you describe is attack resistance of trust metrics. http://usenix.com/publications/library/proceedings/sec98/full_papers/levien/levien.pdf http://usenix.com/publications/library/proceedings/sec98/ful...
Steko 15y ago
All these systems are incredibly vulnerable to capture. Despite their many drawbacks, the best thing peer reviewed journals have going for them is their relative independence.

radarsat1 15y ago

> Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. I see an immediate problem there. There are magnitudes more people _outside_ any given area of expertise than inside. Even small votes add up.

juiceandjuice 15y ago
Ladies and gentleman, I present to you the logarithmic scale.

rlpb 15y ago

Apart from the lack of anonymity, I think that the ideals of such a system are very similar to the ideas of any social voting system (such as Stack Overflow, Slashdot, reddit, HN). The goal is to simply have a rating system that works, right? The trouble is that we still haven't managed to do this well in the technology world. And some of us believe that we need to go back to some level of curation. I'd love to see a system as described that actually works. But until we (the tech world) manage to figure out a system that works, I'm not convinced that scientific publishing will fare any differently.

Joakal 15y ago
The technology world already has it; We have Wikipedia (Edit history), Advogato (Paths), StackOverflow (Rep), etc that encourage personal profiles. You are more valued as a loyal member than one that just started. Unlike Reddit, Slashdot and HN that do not encourage personal profiles. Do you even remember my name? What about other HNers besides pg? Why do you read their posts anyway? They work because it's a form of almost pure democracy being applied and have the same problems of it eg tyranny of the majority. What the scientific model can do is a multitude of above; institution API confirming credentials (Referral), history (paths, did this person do other reviews?), etc. Cheap and still effective peer-review. I'm not sure which kind of government type applies to my suggestion above, but there are many forms of government they can apply. More: https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms_of_government https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms...

rflrob 15y ago

Even cooler would be to have each person (explicitly or implicitly) set other people's vote importance. I can imagine a complicated system wherein voting cliques are detected, and used to feed back into the rankings of what papers show up on your page. Ideally, the voting could be more than just "+1/-1" as well, potentially with a couple different axes, so that I could, for instance, say that I think a paper is important to the field, but likely wrong. Just because a paper is controversial doesn't mean I don't want it to be discussed (for instance, if it introduces a new computational technique, but the input data is flawed somehow).

wickedchicken 15y ago
You're matching an engineering solution to a social problem. Trying to map a paper's social significance to an abstract number is really hard, and in the process you lose a lot of ability to present papers based on taste. Showing me a paper because "these numbers are high and you tend to like these kinds of numbers" is fundamentally different than "you should see this because it is groundbreaking." On the other hand, if we can create a metric for elegance and another for "brings the reader closer to enlightenment," then I'm all for it.
chalst 15y ago
The property you describe is attack resistance of trust metrics. http://usenix.com/publications/library/proceedings/sec98/full_papers/levien/levien.pdf http://usenix.com/publications/library/proceedings/sec98/ful...

Steko 15y ago

All these systems are incredibly vulnerable to capture. Despite their many drawbacks, the best thing peer reviewed journals have going for them is their relative independence.