4 ms·

The problem is that many of those tools are affected negatively by the regression to the mean as well. An average comment, that would never be upvoted during t

The problem is that many of those tools are affected negatively by the regression to the mean as well. An average comment, that would never be upvoted during the early elite, will be upvoted by average people during the mean. Reputation follows.

malandrew 16y ago
The question to be answered then is "what mechanisms of reputation and combinatorial game mechanics can prevent regression towards the mean?" I don't have an answer to this off the top of my head, but succinctly describing the problem as you just did oftentimes can be half the battle.
- jacques_chester 16y ago
  Another formulation: "What system can allow a site to absorb N new users without causing regression to the mean, with the constraint the membership is open?" I'm pretty sure the answer is: no such system exists.
- repsilat 16y ago
  Eigenvalue/SVD upvoting. Comments and articles are ranked for you according to their ranking with similarly voting/submitting/commenting users. The frontpage (depending on how you set it up) could look like Reddit's, based on the majority of users, but to logged-in users it'd be more refined to their tastes. Perhaps you could look at the site from others' perspectives. Defining the model, computing likenesses and finding the appropriate material could be quite resource intensive if you didn't think long and hard about the implementation. (You'd might also want to avoid up and down arrows, because "like" and "dislike" quickly supplant "good comment" and "bad comment".)
  - arghnoname 16y ago
    The very first time I saw a site like this, many years ago, that is what I assumed all of the voting was about. I know a lot more now than I did then why resource wise it is the way it is, but I occasionally think about how one might go about doing a more customized version. Superficially at least, it seems to be a good way to put everyone into a community whose size relates to the commonness of their preferences. I think this would occur as long as comments/articles beneath a certain threshold just weren't visible to a particular user. Unfortunately it seems like a lot of calculating to do per hit, but in thinking about it, it doesn't seem to hurt Netflix too badly.
  - Gormo 16y ago
    If it's too much complexity to track each user's preferences in terms of every other user's, you could also take a Bayesian filtering approach, similar to junk mail controls, and compute a custom front page based on each individual user's prior upvotes/downvotes. In either case, the function of the voting mechanism will ultimately shift from generating the public front page to generating a custom view for each user. At that point, you could re-purpose the public front page as a selection mechanism to draw in the highest-quality users - perhaps by choosing a cohort of existing users to have public-upvote rights, or even by just applying old-fashioned editorial controls.
- eftpotrm 16y ago
  The core issue here (IMHO) can be summed up as: * We all (and I'm aware I'm a pretty young account myself) vote on stories and comments to rank their quality * Newer users vote less in line with the historic ethos because they're less aware of it * All our votes have equal power (the downvote filter here is a crude effort to suppress that but isn't powerful enough to be fully effective) * Network effect means a successful site's growth will be closer to exponential than linear * With exponential growth the new who are unaware of the community standards are quickly able to overpower the existing who are aware Hence almost inevitable degradation in quality which people here are complaining about and which I can partially see myself. Two ways to partially address this spring to mind: * Express vote power as a function of reputation - an upgrade to the downvote system. This has the problem that it provides a direct incentive to align with groupthink and post the easy but commonly held stories and threads so as to attract repuation and so increase one's voting power and.... It also, I think, requires floating point reputation and scoring, and so increases computational load. * Display voting based on a similarity function - 'users who liked this also liked....'. This permits cohorts of users to work en masse without prior coordination to lower the ranking of content they find less interesting / valuable and so get what they want more easily. This means though that it increases fragmentation and reinforces groupthink again by design, which is clearly not ideal. It's also again significantly more computationally expensive. TL;DR: Computation can be thrown at partial improvements but they're expensive to operate, open to gaming and likely to reinforce groupthink. I'm not sure it's socially or technically possible to keep the noisy brats out without severely damaging what attracted the Grand Old Users in the first place. Welcome to social groups...
  - jokermatt999 16y ago
    What about disabling voting until a user reaches a certain amount of karma? Of course, you'd have to give the upvote ability to the original members (hand selected?) so that the system could actually get off the ground. It would be similar to the "invite only" community idea where older users have to actually approve new members, but less restrictive. Perhaps even have upvotes on users instead of just comments. It would be similar to an "invite", but just for allowing them to vote. I'd say also keep the hellbanning that we have here. There will sadly be some false positives, but it seems to work very well for keeping out the worst. Perhaps upvoting users could affect the reputation of the folks doing the upvoting. If you upvote a bad user into the community, your reputation suffers as well. Now, this does nothing to solve the problem of groupthink, and in fact somewhat enforces it. However, I think some amount of groupthink is inevitable if you want to create a strong sense of community. Because you're trying to cluster users around a core set of values, it's probable that they'll think similarly in other ways as well. Edit: I think multidimensional voting helps as well. Ignoring the comments voted "funny" on /. helps to filter out what people liked just because it was amusing. I think "agree/disagree" voting would be useful as a separate thing, because too often that winds up conflicting with upvoting/downvoting for quality.

malandrew 16y ago

The question to be answered then is "what mechanisms of reputation and combinatorial game mechanics can prevent regression towards the mean?" I don't have an answer to this off the top of my head, but succinctly describing the problem as you just did oftentimes can be half the battle.

jacques_chester 16y ago
Another formulation: "What system can allow a site to absorb N new users without causing regression to the mean, with the constraint the membership is open?" I'm pretty sure the answer is: no such system exists.
repsilat 16y ago
Eigenvalue/SVD upvoting. Comments and articles are ranked for you according to their ranking with similarly voting/submitting/commenting users. The frontpage (depending on how you set it up) could look like Reddit's, based on the majority of users, but to logged-in users it'd be more refined to their tastes. Perhaps you could look at the site from others' perspectives. Defining the model, computing likenesses and finding the appropriate material could be quite resource intensive if you didn't think long and hard about the implementation. (You'd might also want to avoid up and down arrows, because "like" and "dislike" quickly supplant "good comment" and "bad comment".)
- arghnoname 16y ago
  The very first time I saw a site like this, many years ago, that is what I assumed all of the voting was about. I know a lot more now than I did then why resource wise it is the way it is, but I occasionally think about how one might go about doing a more customized version. Superficially at least, it seems to be a good way to put everyone into a community whose size relates to the commonness of their preferences. I think this would occur as long as comments/articles beneath a certain threshold just weren't visible to a particular user. Unfortunately it seems like a lot of calculating to do per hit, but in thinking about it, it doesn't seem to hurt Netflix too badly.
- Gormo 16y ago
  If it's too much complexity to track each user's preferences in terms of every other user's, you could also take a Bayesian filtering approach, similar to junk mail controls, and compute a custom front page based on each individual user's prior upvotes/downvotes. In either case, the function of the voting mechanism will ultimately shift from generating the public front page to generating a custom view for each user. At that point, you could re-purpose the public front page as a selection mechanism to draw in the highest-quality users - perhaps by choosing a cohort of existing users to have public-upvote rights, or even by just applying old-fashioned editorial controls.
eftpotrm 16y ago
The core issue here (IMHO) can be summed up as: * We all (and I'm aware I'm a pretty young account myself) vote on stories and comments to rank their quality * Newer users vote less in line with the historic ethos because they're less aware of it * All our votes have equal power (the downvote filter here is a crude effort to suppress that but isn't powerful enough to be fully effective) * Network effect means a successful site's growth will be closer to exponential than linear * With exponential growth the new who are unaware of the community standards are quickly able to overpower the existing who are aware Hence almost inevitable degradation in quality which people here are complaining about and which I can partially see myself. Two ways to partially address this spring to mind: * Express vote power as a function of reputation - an upgrade to the downvote system. This has the problem that it provides a direct incentive to align with groupthink and post the easy but commonly held stories and threads so as to attract repuation and so increase one's voting power and.... It also, I think, requires floating point reputation and scoring, and so increases computational load. * Display voting based on a similarity function - 'users who liked this also liked....'. This permits cohorts of users to work en masse without prior coordination to lower the ranking of content they find less interesting / valuable and so get what they want more easily. This means though that it increases fragmentation and reinforces groupthink again by design, which is clearly not ideal. It's also again significantly more computationally expensive. TL;DR: Computation can be thrown at partial improvements but they're expensive to operate, open to gaming and likely to reinforce groupthink. I'm not sure it's socially or technically possible to keep the noisy brats out without severely damaging what attracted the Grand Old Users in the first place. Welcome to social groups...
- jokermatt999 16y ago
  What about disabling voting until a user reaches a certain amount of karma? Of course, you'd have to give the upvote ability to the original members (hand selected?) so that the system could actually get off the ground. It would be similar to the "invite only" community idea where older users have to actually approve new members, but less restrictive. Perhaps even have upvotes on users instead of just comments. It would be similar to an "invite", but just for allowing them to vote. I'd say also keep the hellbanning that we have here. There will sadly be some false positives, but it seems to work very well for keeping out the worst. Perhaps upvoting users could affect the reputation of the folks doing the upvoting. If you upvote a bad user into the community, your reputation suffers as well. Now, this does nothing to solve the problem of groupthink, and in fact somewhat enforces it. However, I think some amount of groupthink is inevitable if you want to create a strong sense of community. Because you're trying to cluster users around a core set of values, it's probable that they'll think similarly in other ways as well. Edit: I think multidimensional voting helps as well. Ignoring the comments voted "funny" on /. helps to filter out what people liked just because it was amusing. I think "agree/disagree" voting would be useful as a separate thing, because too often that winds up conflicting with upvoting/downvoting for quality.