5 ms·

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

by Joakal 16y ago

danvet 16y ago
On a hunch: Run Pagerank (the original thing, i.e. just the stable distribution of the random walk) on the graph of who-upvotes-whom. Then tally upvotes weighted with the upvoters pagerank. It's probably safe to assume that the pagerank of a given user changes rather slowly, so this shouldn't be a too big problem to actually compute (and needs to be update at most every few days). Also, to favour recent history a bit one could decay the wheight of older upvotes in the who-upvotes-whom graph. Now that upvotes are non-public information, people can't know their own rank (at least not easily), so this looks at least halfway resistant to gaming.
- chalst 16y ago
  Advogato has used an attack-resistant trust metric with similar features; Raph Levien called such systems attack-resistent trust metrics. I'm sure they would do the task you describe successfully. Check out: 1. http://www.levien.com/free/tmetric-HOWTO.html http://www.levien.com/free/tmetric-HOWTO.html 2. http://www.advogato.org/person/chalst/diary/189.html http://www.advogato.org/person/chalst/diary/189.html
- bermanoid 16y ago
  Whoops, missed a lot of responses here, sorry about that! Yes, vanilla Pagerank (where the "sites" are accounts and "links" are upvotes) is the first thing that came to my mind, mainly because it's been pretty well battle tested; running it on the comment graph is actually a much simpler problem than running it behind a search engine because you don't need the additional refinements based on search keyword. A comment's Pagerank is, by itself, enough to set an ordering. If that's too much work, though, even a simple vote weighting based on some function of the upvoter's karma would be a decent approximation. Say every user's votes had an impact based on some sigmoid function of their karma (maybe a Gompertz function?), tuned so that the plateau is hit for the top 10% of users or something like that.
jamesbritt 16y ago
Your last suggestion seems to make things less democratic understandably, but how would it be implemented? Weighted votes based on karma. Or restrict the number of votes a user can cast based on karma or account age. A problem with these kinds of things is determining what does or does not work and picking up on unintended side-effects, and doing so in a timely manner. For example, if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters? How long would it take to see this? What if by the time you recognize an undesired side effect you've already sent the site down the road of ruin (or something)? For example, how long will pg keep comment vote hidden? How many people have found this to be sufficiently detrimental that they have, or soon will, leave HN and not come back? How do you craft meaningful site experiments like this while keeping risk to a minimum?
- Joakal 16y ago
  I suggest similar to Reddit's anti-spam mechanism, fuzz the voting but give the impression that karma is somewhat important.
  - andreaja 16y ago
    Maybe all we need is 'bad, average, good, great' to be shown alongside the comment. Or something along the lines of what slashdot do with allowing you to vote for comments as 'insightful' or 'funny' etc.
- bullseye 16y ago
  >> if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters? I suppose that depends on whether or not they are on the site to read submitted news articles and comments or to play the voting and karma game. I rarely vote (or comment for that matter), but I get value from the content. For that reason, I have continued to consistently visit the site for a couple of years.
alanfalcon 16y ago
I like the idea of upvotes and downvotes being multiplied by your avg score. tptacek's votes would be worth 8.85 at the moment, bermanoid's 3.5, and mine 2.92. Newbies would come in at 1.
- epochwolf 16y ago
  Wouldn't you want newbies at 0.01 (or some other value below 1) until they have a reliable score?
- ignifero 16y ago
  That would be a violation of the democratic principle behind social news and would incentivize people to game their scores. Members have different interests and expertise, so having a high avg score doesn't mean they are experts in everything (Abuse of power - OMG this is getting more intricate than i thought)
  - jbooth 16y ago
    You could take the log of any score > 1 to reduce the incentive to game the system.. that way someone who relentlessly karma whores won't have that much more influence than a mildly uprated commenter, but both will have more influence than a noob.
    - ignifero 16y ago
      but then a mob of noobs could outrank two wise elders.
      - bermanoid 16y ago
        Well, with the situation we're in now, only three noobs (or one noob with three accounts) can outrank two wise elders. As to your gp comment, yes, incentivizing karma gains could be a problem, but that's the tradeoff that's got to be considered. I'm not sure what the best tradeoff is; personally, I'd probably go with something like Pagerank for a while and see if the comment quality (above the fold, at least) improves, but perhaps pg has a desire to keep things more neutral and fair, I'm not sure.
Super_Jambo 16y ago
I've actually implemented a system like this on isdaily (a news site I'm building.) I came to the conclusion that using Karma as a score is a rather bad idea as it assumes that people who get voted up have better judgement: "If you post well you must vote well." But I'm sceptical that this is the case, it's easy to see someone like patio11 has both excellent judgement and a high karma score. But vote power = karma probably means you're giving a casting vote to patio11 if he turns up but mainly you're giving vote power to the people who post the most. The implementation I have is basically: "If you made hard decisions in the past and voted well, you get more vote power."
- Gormo 16y ago
  So in your implementation, you gain voting weight by voting on the posts that have themselves accumulated the highest vote counts? In order for this to work, you'd have to hide vote totals on all posts and comments, which, as jamesbritt pointed out above, eliminates a feedback mechanism that users may find desirable.
  - Super_Jambo 15y ago
    I do hide the totals. However you gain vote power for difficult decisions. So the most vote power is assigned to someone who votes early on a contentious story which later becomes popular.

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

by Joakal 16y ago

Your last suggestion seems to make things less democratic understandably, but how would it be implemented?

On a hunch: Run Pagerank (the original thing, i.e. just the stable distribution of the random walk) on the graph of who-upvotes-whom. Then tally upvotes weighted with the upvoters pagerank. It's probably safe to assume that the pagerank of a given user changes rather slowly, so this shouldn't be a too big problem to actually compute (and needs to be update at most every few days). Also, to favour recent history a bit one could decay the wheight of older upvotes in the who-upvotes-whom graph. Now that upvotes are non-public information, people can't know their own rank (at least not easily), so this looks at least halfway resistant to gaming.

chalst 16y ago
Advogato has used an attack-resistant trust metric with similar features; Raph Levien called such systems attack-resistent trust metrics. I'm sure they would do the task you describe successfully. Check out: 1. http://www.levien.com/free/tmetric-HOWTO.html http://www.levien.com/free/tmetric-HOWTO.html 2. http://www.advogato.org/person/chalst/diary/189.html http://www.advogato.org/person/chalst/diary/189.html
bermanoid 16y ago
Whoops, missed a lot of responses here, sorry about that! Yes, vanilla Pagerank (where the "sites" are accounts and "links" are upvotes) is the first thing that came to my mind, mainly because it's been pretty well battle tested; running it on the comment graph is actually a much simpler problem than running it behind a search engine because you don't need the additional refinements based on search keyword. A comment's Pagerank is, by itself, enough to set an ordering. If that's too much work, though, even a simple vote weighting based on some function of the upvoter's karma would be a decent approximation. Say every user's votes had an impact based on some sigmoid function of their karma (maybe a Gompertz function?), tuned so that the plateau is hit for the top 10% of users or something like that.

jamesbritt 16y ago

Your last suggestion seems to make things less democratic understandably, but how would it be implemented? Weighted votes based on karma. Or restrict the number of votes a user can cast based on karma or account age. A problem with these kinds of things is determining what does or does not work and picking up on unintended side-effects, and doing so in a timely manner. For example, if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters? How long would it take to see this? What if by the time you recognize an undesired side effect you've already sent the site down the road of ruin (or something)? For example, how long will pg keep comment vote hidden? How many people have found this to be sufficiently detrimental that they have, or soon will, leave HN and not come back? How do you craft meaningful site experiments like this while keeping risk to a minimum?

Joakal 16y ago
I suggest similar to Reddit's anti-spam mechanism, fuzz the voting but give the impression that karma is somewhat important.
- andreaja 16y ago
  Maybe all we need is 'bad, average, good, great' to be shown alongside the comment. Or something along the lines of what slashdot do with allowing you to vote for comments as 'insightful' or 'funny' etc.
bullseye 16y ago
>> if "young" accounts have very limited votes available will new users be less likely to stick around long enough to become more active voters? I suppose that depends on whether or not they are on the site to read submitted news articles and comments or to play the voting and karma game. I rarely vote (or comment for that matter), but I get value from the content. For that reason, I have continued to consistently visit the site for a couple of years.

alanfalcon 16y ago

I like the idea of upvotes and downvotes being multiplied by your avg score. tptacek's votes would be worth 8.85 at the moment, bermanoid's 3.5, and mine 2.92. Newbies would come in at 1.

epochwolf 16y ago
Wouldn't you want newbies at 0.01 (or some other value below 1) until they have a reliable score?
ignifero 16y ago
That would be a violation of the democratic principle behind social news and would incentivize people to game their scores. Members have different interests and expertise, so having a high avg score doesn't mean they are experts in everything (Abuse of power - OMG this is getting more intricate than i thought)
- jbooth 16y ago
  You could take the log of any score > 1 to reduce the incentive to game the system.. that way someone who relentlessly karma whores won't have that much more influence than a mildly uprated commenter, but both will have more influence than a noob.
  - ignifero 16y ago
    but then a mob of noobs could outrank two wise elders.
    - bermanoid 16y ago
      Well, with the situation we're in now, only three noobs (or one noob with three accounts) can outrank two wise elders. As to your gp comment, yes, incentivizing karma gains could be a problem, but that's the tradeoff that's got to be considered. I'm not sure what the best tradeoff is; personally, I'd probably go with something like Pagerank for a while and see if the comment quality (above the fold, at least) improves, but perhaps pg has a desire to keep things more neutral and fair, I'm not sure.

Super_Jambo 16y ago

I've actually implemented a system like this on isdaily (a news site I'm building.) I came to the conclusion that using Karma as a score is a rather bad idea as it assumes that people who get voted up have better judgement: "If you post well you must vote well." But I'm sceptical that this is the case, it's easy to see someone like patio11 has both excellent judgement and a high karma score. But vote power = karma probably means you're giving a casting vote to patio11 if he turns up but mainly you're giving vote power to the people who post the most. The implementation I have is basically: "If you made hard decisions in the past and voted well, you get more vote power."

Gormo 16y ago
So in your implementation, you gain voting weight by voting on the posts that have themselves accumulated the highest vote counts? In order for this to work, you'd have to hide vote totals on all posts and comments, which, as jamesbritt pointed out above, eliminates a feedback mechanism that users may find desirable.
- Super_Jambo 15y ago
  I do hide the totals. However you gain vote power for difficult decisions. So the most vote power is assigned to someone who votes early on a contentious story which later becomes popular.