The problem with most online reviews is that they usually only collect reviews from those who are motivated to submit a review. This usually skews reviews towar

The problem with most online reviews is that they usually only collect reviews from those who are motivated to submit a review. This usually skews reviews towards the 1 or 5 star ends of the spectrum because 3-star reviewers aren't really motivated to go through the hassle of submitting a review, unless they're the type of person who obsessively reviews everything (think Yelp! Elite). The only real way to get past this is to know when someone is a candidate to review something and solicit reviews from them. Amazon reviews fall into this category and it's the reason why they're pretty reliable. I'd bet they're getting 5-10% conversion (I've worked on systems that got over 30%, but different types of reviewable products have different conversion rates) on every "Did product X meet your expectations?" email they send. They've got their die hards that review everything and who's reviews everyone finds helpful, but the star rating will be largely determined by the larger group of people that write 1-2 sentences and submit. But without closing the loop and attempting to get reviews from every possible reviewer, ratings are far less useful.