3 ms·

I've been toying with the idea of adapting the Elo rating system from chess to a social site's user score. The basic idea from chess is that you assume someone

I've been toying with the idea of adapting the Elo rating system from chess to a social site's user score. The basic idea from chess is that you assume someone has an intrinsic skill in some arbitrary units (I think you start out around 1,000 and a 2400+ is senior master) and that we can only infer that skill from how they perform in tournaments. Furthermore, your skill in any particular game is assumed to be normally distributed about that intrinsic skill value. It is also assumed that your skill will only change slowly over time (we don't have to pin down a moving target). So the mechanics of this system are basically that when you compete with someone that is much better than you and win your score should be adjusted upwards. Conversely, if you lose to someone worse than you, your score will be adjusted downward. Adjustments are done in batches, say after a player has finished a tournament (I use five voting events between readjustment). My application to a user karma system would be analogous in that a "match" or "competition" would happen every time someone directly rates someone else (stories may be exempt). A high-karma user upvoting a low-karma user would increase the low-karma user's karma and decrease the high-karma user's karma. If there are two users with equal karma and one up or downvotes the other. the number of points exchanged will be minimal (the gain/loss scales with the difference in karma). The idea is that your ideas are competing with the opinions of the high-ranked users. The assumption is that new users will have bad ideas and that high-karma users will downvote them. A downvote from a high-karma user is expected and doesn't cost the new user very much (they are expected to "lose" to them often). I've sketched out some of the functions here: http://paste.lisp.org/display/76193 http://paste.lisp.org/display/76193 EDIT: grammar fix, typed this before coffee

calambrac 18y ago
That doesn't seem like a very nice assumption, that new users will have bad ideas, and if old users lose karma every time they vote up a new user, how can you ever know if the new user genuinely has nothing to offer, or if old users are just wary of decreasing their score, and therefore, influence? (i.e., I have lots of karma, and I know if I upvote this comment that I like, it will increase their score quite a bit - I have influence now - but it will also lower my karma, and I'll have less influence over the next comment I see which I may like more...) This is what happens in chess: highly-ranked players just simply won't play new or lower-ranked players at all. It's not a bad thing there, but then again, it is a competition. A social site isn't, though.
- twopoint718 18y ago
  Oh I agree completely that it isn't a nice assumption; it is just a simplifying one (like the proverbial spherical cow) that's mostly wrong. But I don't think it is entirely wrong, and the ways that it is right may be useful, as an experiment. Maybe I shouldn't have used the word bad. But new users may not know the established traditions of a group, maybe they haven't even bothered to read the FAQ. The articles that got me thinking along these lines were: http://www.shirky.com/writings/group_enemy.html http://www.shirky.com/writings/group_enemy.html and http://www.cc.gatech.edu/classes/AY2001/cs6470_fall/LTAND.html http://www.cc.gatech.edu/classes/AY2001/cs6470_fall/LTAND.ht... . The gist is that an important group function is to defend itself against (new) users. But I think that what it'll do is to make votes matter because they cost something. Mostly, I'm interested to see what the implications would be.
  - calambrac 18y ago
    Okay, but why should votes cost more for a user with more karma? I would think the opposite should be true, that if karma is an indicator of a user's value to the site, someone with more karma should pay less to vote up a newer user.
    - twopoint718 18y ago
      The way that I'm thinking about it is this: that it is relatively unusual, from the standpoint of the community, for a high-karma user to upvote a low-karma user. Karma is a measure of the norms of the community, parceled out to users.

calambrac 18y ago

That doesn't seem like a very nice assumption, that new users will have bad ideas, and if old users lose karma every time they vote up a new user, how can you ever know if the new user genuinely has nothing to offer, or if old users are just wary of decreasing their score, and therefore, influence? (i.e., I have lots of karma, and I know if I upvote this comment that I like, it will increase their score quite a bit - I have influence now - but it will also lower my karma, and I'll have less influence over the next comment I see which I may like more...) This is what happens in chess: highly-ranked players just simply won't play new or lower-ranked players at all. It's not a bad thing there, but then again, it is a competition. A social site isn't, though.

twopoint718 18y ago
Oh I agree completely that it isn't a nice assumption; it is just a simplifying one (like the proverbial spherical cow) that's mostly wrong. But I don't think it is entirely wrong, and the ways that it is right may be useful, as an experiment. Maybe I shouldn't have used the word bad. But new users may not know the established traditions of a group, maybe they haven't even bothered to read the FAQ. The articles that got me thinking along these lines were: http://www.shirky.com/writings/group_enemy.html http://www.shirky.com/writings/group_enemy.html and http://www.cc.gatech.edu/classes/AY2001/cs6470_fall/LTAND.html http://www.cc.gatech.edu/classes/AY2001/cs6470_fall/LTAND.ht... . The gist is that an important group function is to defend itself against (new) users. But I think that what it'll do is to make votes matter because they cost something. Mostly, I'm interested to see what the implications would be.
- calambrac 18y ago
  Okay, but why should votes cost more for a user with more karma? I would think the opposite should be true, that if karma is an indicator of a user's value to the site, someone with more karma should pay less to vote up a newer user.
  - twopoint718 18y ago
    The way that I'm thinking about it is this: that it is relatively unusual, from the standpoint of the community, for a high-karma user to upvote a low-karma user. Karma is a measure of the norms of the community, parceled out to users.