3 ms·
Scatter plots are definitely more informative, once one gives them a couple of minutes to get used to them. However, I think you're shooting at the wrong target
by CodeMage 14y ago
Scatter plots are definitely more informative, once one gives them a couple of minutes to get used to them. However, I think you're shooting at the wrong target and your solution would exacerbate the root problem: bias.
The first time I really noticed the problem was when I published my own Flash game on Kongregate and started paying closer attention to the ratings. That led me to examine my own rating habits and I conjectured that is probably what happens to everyone else.
The bias I'm talking about is caused by the fact that most people can't be bothered to rate something. Most people only rate something when there's a powerful impulse to do so, so most of the votes will be 5 stars or 1 star. The 4-star ratings come from people who liked something enough to be moved to rate it, but not enough to gush about it; note that the group of people who makes that distinction is already substantially smaller than the 5- and 1-star reviewers. The rest comes from a very small minority, most of whom are people who didn't have anything better to do at that moment and decided to spend some time rating, but don't do it on the regular basis.
By the way, I realize that this is just a conjecture, but from what I've seen so far, it seems to be pretty accurate.
I think that introducing an additional axis will only exacerbate this, by raising the bar for rating. If the act of rating starts demanding more effort, you'll get a distribution that is even more skewed than now.
The two improvements I would like to see are:
1. a system that infers ratings from users' actions
2. better mechanisms for gauging the relevance of someone's review/rating based on my preferences/tastes
The first would help reduce the bias and the second would help me extract more useful information from the biased dataset.
- scottmcf 14y agoObviously, it doesn't explain everything from your example, but Kongregate rewards users for voting for games (1 point per rating in their levelling scheme). This will have some impact on how people vote.
- awakeasleep 14y agoI've noticed the same problem, where people tend to rate a 5 or a 1. As a benefit, I've noticed I can usually find the best reviews on Amazon by looking for 3 star ratings, and to a lesser extend 4 and 2 stars. People who rate a 3 have looked at pros and cons of the product, and generally compare to similar goods. 3 star reviews usually provide FAR more information than glowing or glowering 4 and 1's.
- tolos 14y agoSimilar to what you said, where YouTube found most people only voted 5 stars, with 1 and 4 stars being used much less frequently: http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html http://youtube-global.blogspot.com/2009/09/five-stars-domina...