2 ms·
Kuro5hin's moderation and reputation system had some novel-for-the time (and still comparatively rare) features, though it ultimately suffers much the same fail
by dredmorbius 6y ago
Kuro5hin's moderation and reputation system had some novel-for-the time (and still comparatively rare) features, though it ultimately suffers much the same failure as most other crowdsourced moderation systems, and ultimately doesn't work under sustained attack or concerted effort.
For those unfamiliar, Kuro5hin, inspired by Slashdot, was an online discussion site where individuals would submit items, along with a (hopefully) brief description, or an original essay. There was no editorial queue (as Slashdot had and has). Both posts and comments could be rated on a five-point scale by individuals, and the ultimate score landed in the bound 1-5 (later extended to 0 or -1 with flags). Moderation converged on a mean with time, at least in theory. Users "mojo" (karma) would follow similarly.
This avoids both the runaway endless votes problem of sites such as Reddit (or Hacker News), and the behaviour of Slashdot in which moderation would rise or fall but never converge and was artificially capped at 5 points. On Kuro5hin, the 100th moderation score was weighted as 1/100. On Slashdot, the 100th moderation score had a weight of 1.
If this sounds something like a star-based review system such as Rotten Tomatoes, or Amazon ... it's because it is. And ultimately the same problems apply:
- Sybil attacks in which sockpuppet or shill accounts are created to sway moderation / review scores are quite possible.
- If the site fails to attract (or motivate) consciencious moderators, the moderation quality still falls.
- Information on divergent (strongly bimodal) rating is lost. 10 moderations split between 5 '1's and 5 '5's look the same as 10 '3' ratings. The information contained is not the same. (Summarising complex data is, it turns out, complex.)
- There's still no distinction between popular and truthful content. Or between "this is true" and "I disagree with this" (or don't like it, or the submitter looks funny, or whatever).
The K5 system was interesting, but ultimately not a magic bullet. The site itself failed, though for complex reasons.
- tunesmith 6y agoAh, interesting. I could have sworn K5 had a vouching system, where the strength of a rating was partially dependent on the rating of the person doing the rating.
- dredmorbius 6y agoNot AFAIA, though that's possible. The original concept didn't have that. Another approach was tried by Advogato, which had a "trust root" (ultimately Raph Levien). It was based around individuals rather than content and ... that also has issues. Nothing pathologically bad in the case of Advogato, though the site largely languished. For Levien's concept, see http://www.levien.com/thesis/compact.pdf http://www.levien.com/thesis/compact.pdf