5 ms·

I find the 5 star system on Amazon useful. I don't always look at the average rating, but rather the distribution graph of ratings. For example, one common sh

I find the 5 star system on Amazon useful. I don't always look at the average rating, but rather the distribution graph of ratings. For example, one common shape looks like this: 5 stars: ||||||||||| 4 stars: |||||||| 3 stars: |||| 2 stars: || 1 star: ||||||| There's way more 1 star than 2 or 3. That usually indicates the product shows up DOA for a lot of people, or is otherwise probabilistically totally unsuitable for some group of people. If the number of star ratings follows a more normal curve like the following, then I'm more confident in the quality of the product, even if both have an average 4 star rating: 5 stars: ||||||| 4 stars: |||||||| 3 stars: |||| 2 stars: || 1 star: |

mashmac2 13y ago
This would make a fascinating study of reliability - can you determine statistically if a product will be reliable based on rating distribution. You could study Amazon, App Store ratings, etc. Cool insight!
- sanskritabelt 13y ago
  It's been done, and I've seen it as a worked example in introduction to Bayesian stats pages, maybe some of which were posted on HN.
- arh68 13y ago
  I wouldn't take any rating metric seriously unless it accounted for how long they've owned it. Plus there's the difference between "I've owned 12 lawnmowers and this one's pretty okay, only broke down after 1.5yrs, 3 stars" and "this is my first lawnmower, used it once, it's amazing, never going back to cutting my lawn with scissors, 5 stars". Plus niche sites sometimes have better reviews for the same products (newegg). All that said, there is information to be harvested in some of these reviews.
  - ludzone 13y ago
    Good point about niche sites and the length of ownership. I've noticed that a number of starts on products like top rated power supplies goes down after a year an a half or so. While you might be getting a newly released power supply with 500 reviews averaging closer to 5 stars, after 2 years, you will notice that same product with 1600 reviews averaging at below 4 stars, because owners probably came back and updated their reviews. It would be interesting to see a trend line of change in sentiment about a products over time, using existing data.
santadays 13y ago
The other common pattern being shill reviews: 5 stars: |||||||||||| 4 stars: || 3 stars: |||| 2 stars: ||||| 1 star: ||||||||
xtracto 13y ago
A lot of times, the 1-star ratings are only "knee-jerk reactions". The 5-star ratings are sometimes added by people related to the product (or people who for some reason have an agenda). I usually drop those two and focus on the 2-4 reviews.
- panacea 13y ago
  Who rates something 3? Why bother to interact with a UI to express utter meh?
thaumasiotes 13y ago
> most people think in absolutes: Either something is the bee's knees or it's fried shit on a biscuit. (Yes, this is from two levels up.) There was a fascinating post to the OKCupid blog discussing the distribution of attractiveness ratings (also on a 1-5 scale). Ratings of women by men looked like this: 5: |||| 4: |||||||| 3: |||||||||| 2: |||||||| 1: |||| Ratings of men by women looked like this: 5: || 4: || 3: ||| 2: ||||||| 1: |||||||||||||||||||| EDIT: As you might expect when working from memory, I got some details wrong. Post is here: http://blog.okcupid.com/index.php/your-looks-and-online-dating/ http://blog.okcupid.com/index.php/your-looks-and-online-dati... Obvious corrections: Scale is 0-5, not 1-5 Curve for women is not as steep as depicted in my comment. However, the modal rating is 1, followed by 0. In the words of the post, "women rate an incredible 80% of guys as worse-looking than medium. Very harsh."

mashmac2 13y ago

This would make a fascinating study of reliability - can you determine statistically if a product will be reliable based on rating distribution. You could study Amazon, App Store ratings, etc. Cool insight!

sanskritabelt 13y ago
It's been done, and I've seen it as a worked example in introduction to Bayesian stats pages, maybe some of which were posted on HN.
arh68 13y ago
I wouldn't take any rating metric seriously unless it accounted for how long they've owned it. Plus there's the difference between "I've owned 12 lawnmowers and this one's pretty okay, only broke down after 1.5yrs, 3 stars" and "this is my first lawnmower, used it once, it's amazing, never going back to cutting my lawn with scissors, 5 stars". Plus niche sites sometimes have better reviews for the same products (newegg). All that said, there is information to be harvested in some of these reviews.
- ludzone 13y ago
  Good point about niche sites and the length of ownership. I've noticed that a number of starts on products like top rated power supplies goes down after a year an a half or so. While you might be getting a newly released power supply with 500 reviews averaging closer to 5 stars, after 2 years, you will notice that same product with 1600 reviews averaging at below 4 stars, because owners probably came back and updated their reviews. It would be interesting to see a trend line of change in sentiment about a products over time, using existing data.

santadays 13y ago

The other common pattern being shill reviews: 5 stars: |||||||||||| 4 stars: || 3 stars: |||| 2 stars: ||||| 1 star: ||||||||

xtracto 13y ago

A lot of times, the 1-star ratings are only "knee-jerk reactions". The 5-star ratings are sometimes added by people related to the product (or people who for some reason have an agenda). I usually drop those two and focus on the 2-4 reviews.

panacea 13y ago
Who rates something 3? Why bother to interact with a UI to express utter meh?

thaumasiotes 13y ago

> most people think in absolutes: Either something is the bee's knees or it's fried shit on a biscuit. (Yes, this is from two levels up.) There was a fascinating post to the OKCupid blog discussing the distribution of attractiveness ratings (also on a 1-5 scale). Ratings of women by men looked like this: 5: |||| 4: |||||||| 3: |||||||||| 2: |||||||| 1: |||| Ratings of men by women looked like this: 5: || 4: || 3: ||| 2: ||||||| 1: |||||||||||||||||||| EDIT: As you might expect when working from memory, I got some details wrong. Post is here: http://blog.okcupid.com/index.php/your-looks-and-online-dating/ http://blog.okcupid.com/index.php/your-looks-and-online-dati... Obvious corrections: Scale is 0-5, not 1-5 Curve for women is not as steep as depicted in my comment. However, the modal rating is 1, followed by 0. In the words of the post, "women rate an incredible 80% of guys as worse-looking than medium. Very harsh."