3 ms·
Maybe? Reputation systems can devolve into rewarding groupthink. It's a classic "you get what you measure" conundrum, where once it becomes clear that an opinio
by brookst 4y ago
Maybe? Reputation systems can devolve into rewarding groupthink. It's a classic "you get what you measure" conundrum, where once it becomes clear that an opinion / phrase / meme is popular, it's easy to farm reputation by repeating it.
I like your comment about "track record of insightful, levelheaded conversation", but that introduces another abstraction. Who measures insight or levelheadedness, and how to avoid that being gamed?
I general I agree that reputation is an interesting and potentially important signal, I'm just not sure I've ever seen an implementation that doesn't cause a lot of the problems it's trying to solve. Any good examples?
- mjjjjjjj 4y agoThis is pure hypothetical, but I bet Reddit could derive an internal reputational number that is a combination of both karma (free and potentially farmable) and awards (that people actually pay for or that are scarce and shows what they value) that would be a better signal to noise ratio than just karma alone.
- pixl97 4y agoSo a wealthy bot farmer rules this system?
- duckmysick 4y agoIt could work like Google search where link farms have less weight than higher quality pages. Yes, you can still game it yet it will be harder to do so.
- dbrueck 4y agoYeah, definitely potential for problems and downsides. And I don't know of any implementations that have gotten it right. And to some degree, I imagine all such systems (online or not) can be gamed, so it's also important for the designers of such a system to not try to solve every problem either. And maybe you do have some form of moderation, but not in the sense of moderation of your agreement/disagreement with ideas but moderation of behavior - like a debate moderator - based on the rules of the community. Your participation in a community would involve reading, posting as desired once you've been in a community for a certain amount of time, taking a turn at evaluating N comments that have been flagged, and taking a turn at evaluating disputes about evaluations, with the latter 2 being spread around so as to not take up a lot of time (though, having those duties could also reinforce your investment in a community). The reach/visibility of your posts would be driven off your reputation in that community, though people reading could also control how much they see too - maybe I only care about hearing from more established leaders while you are more open to hearing from newer / lower reputation voices too. An endorsement from someone with a higher reputation counts more than an endorsement from someone who just recently joined, though not so huge of a difference that it's impossible for new ideas to break through. As far as who measures, it's your peers - the other members of the community, although there needs to be a ripple effect of some sort - if you endorse bad behavior, then that negatively effects your reputation. If someone does a good job of articulating a point, but you ding them simply because you disagree with that point, then someone else can ding you. If you consistently participate in the community duties well, it helps your reputation. The above is of course super hand-wavy and incomplete, but something along those lines has IMO a good shot of at least being a better alternative to some of what we have today and, who knows, could be quite good.
- brookst 4y ago> Your participation in a community would involve reading, posting as desired once you've been in a community for a certain amount of time, taking a turn at evaluating N comments that have been flagged, and taking a turn at evaluating disputes about evaluations, with the latter 2 being spread around so as to not take up a lot of time (though, having those duties could also reinforce your investment in a community). This is an interesting idea, and I'm not sure it even needs to be that rigorous. Active evaluations are almost a chore that will invite self-selection bias. Maybe we use sentiment analysis/etc to passively evaluate how people present and react to posts? It'll be imperfect in any small sample, but across a larger body of content, it should be possible to derive metrics like "how often does this person compliment a comment that they also disagree with" or "relative to other people, how often do this person's posts generate angry replies", or even "how often does this person end up going back and forth with one other person in an increasingly angry/insulting style"? It still feels game-able, but maybe that's not bad? Like, I am going to get such a great bogus reputation by writing respectful, substantive replies and disregarding bait like ad hominems! That kind of gaming is maybe a good thing. One fun thing is this could be implemented over the top of existing communities like Reddit. Train the models, maintain a reputation score externally, offer an API to retrieve, let clients/extensions decide if/how to re-order or filter content.
- fblp 4y agoGoogle search is an example of ude of site reputation (search ranking) driven by where backlinks and various other site quality metrics. I would also say the Facebook feed also ranks based on the reputation and relevance of the poster of the content.
- pixl97 4y agoIs google supposed to be a positive or negative example here, that is with all the recent complaints about SEO spam and search quality dropping?