3 ms·
> BeerAdvocate, of all things, takes on this problem with interesting meta-info like "reviewer's average distance from consensus". There's a lot to be said for
by rlucas 8y ago
> BeerAdvocate, of all things, takes on this problem with interesting meta-info like "reviewer's average distance from consensus".
There's a lot to be said for intelligently dealing with ratings and their metadata in ways like this.
For example, the Kappa statistic: https://en.wikipedia.org/wiki/Inter-rater_reliability https://en.wikipedia.org/wiki/Inter-rater_reliability
Presenting crowdsourced ratings as mere averages -- or really any collapse onto a single scalar -- is a big step up from no info at all, but it's hardly the best one can do even with well-known statistical techniques.
- Bartweiss 8y agoYep, I really wish sites would either give me a histogram of raw scores, provide more thoughtful interpretations, or both. Histograms are a screamingly obvious way of distinguishing "mediocre" from "some good some bad", which is one of the most common needs with things like Amazon products. But beyond that, there's so much more to be done. You can weight or shift scores by reviewer's average, reviewer's average distance from consensus, or a dozen other things. A one-star review from someone who uses Yelp exclusively to call out bad experiences is relevant, but a one-star review from someone who often gives 4-5 is far more interesting. Maybe the weirdest thing is that a lot of this is done to catch fake/paid reviewers, but it's not extended to providing clearer info overall. Even the fight against fake reviews would be much achievable if it was shifted from a binary "take down or don't" to a more flexible approach to maximizing review value.