4 ms·

User contributed ratings have, it seems, mostly outlived their usefulness. It isn't that things can't be reviewed, but ratings are like anything else in life,

by taeric 3y ago

User contributed ratings have, it seems, mostly outlived their usefulness. It isn't that things can't be reviewed, but ratings are like anything else in life, in that they have deaths. You want me to trust ratings more, let me order them by date. Oddly, Steam does a good job with this. Two ratings, recent and all time. Google, amusingly, only shows the all time on the search page, but the recent reviews are very relevant. Does this have to apply to all things? Maybe not, but manufacturing changes. needs of the time change. General expectations change. A 4 star rating from a decade is of much less reliability than one today. Even if it is critically a good rating.

stronglikedan 3y ago
My system is to just focus on the 1 & 2 star ratings, and look for trends, specifically. It takes a bit of work, since most of those reviews are just nonsense, but eventually true flaws in the product will be exposed through repetition. Due to the amount of work, I typically only reserve this method for big ticket items that would be difficult to return, such as appliances.
- taeric 3y ago
  Oddly, appliances are the area I would trust user reviews the least. It will be dominated by luck of experience on who will post. And with how variable manufacturing is for most, the exact same model sold on the other side of the nation is as likely to be a different beast as not. Not even considering last year's version of it. Still, I don't discard the idea here. If there are features of it that every low star mentions, I think I agree that I would trust the feature is bad. I can't think of often that has happened. Will try to pay more attention for it.
deleted 3y ago
[deleted]
rjh29 3y ago
Steam does that well, but its recommended/not recommended system is a dark pattern IMO. People tend to choose recommended even for games they don't like on the basis that someone else might like it, or it might improve in the future. Virtually every popular game has 'Overwhelmingly Positive' or similar. To get useful data you need to read the not recommended reviews in detail. A more honest system would be 'Bad', 'Average' and 'Fantastic' - then I bet you'd get a ton more 'average' reviews.
- taeric 3y ago
  Mostly agree, but I wouldn't call it a dark pattern. The problem is you are asking for two things. Do you want popular recommendations? Or do you want critical reviews? Those are very different and very hard to house together. This is why I mostly view the problem as being with user provided ratings. They basically fall into the same realm as all big data. Tons of noise, little signal, and easy to misinterpret.
deleted 3y ago
[deleted]
nonameiguess 3y ago
I think this highly depends on what is being rated. Non-durable consumer goods, sure, there is very little assurance that what you're getting now is even what you would have gotten ten years ago. Processes and inputs both change. A previously reputable brand might have been purchased by private equity and enshittified on purpose. On the other hand, if you're reading reviews of Moby Dick, I don't think the perceived quality or relevance of the writing is going to change much on the scale of decades. Maybe it starts to get archaic at some point, but people still get a lot out of reading Homer. If it's something like Principia Mathematica, that might honestly be timeless as long as human-like creatures exist who can read and understand math and physics.
- taeric 3y ago
  Fair. Books probably are a bit more evergreen on the reviews. Though, I am guessing that is highly variable on the genre? The Steam approach would probably still work rather well. No need to delete old reviews, but having recent ones can still be good to see.
_ea1k 3y ago
I've found that sorting Amazon reviews by date works pretty well. It isn't uncommon to see all 5 star reviews for a while and mostly 1 star reviews in the more recent history.
- madcaptenor 3y ago
  Seems like it wouldn't be too hard for Amazon to weight recent reviews more heavily.
  - malfist 3y ago
    It wouldn't take too much effort for amazon to make their marketplace better in general, but they make money off of it being bad. The incentives aren't aligned, at least until reputation is hurt badly enough to cost them more profit than the short term gains they make by allowing false reviews and false advertising.
  - _ea1k 3y ago
    They could, but honestly that might be risky. It might just encourage the fake reviewers to go at a slower but more steady pace to beat the system.
robomartin 3y ago
> You want me to trust ratings more, let me order them by date. Interesting idea. Having been involved with multiple third party sellers on Amazon, my extension of your thought is to have the ability to display a graph of all five rating levels over time. This makes rating manipulation visible at both ends of the scale. I know of people who, for example, sent out free samples of their product to a thousand people or more with a suggestion to post a four or five star review (after which they would get more goodies for free). I also know of sellers who were attacked by either bots or a mob (in one case a competitor hired a team outside the US) who post hundreds of 1-star ratings and negative reviews. If you could see a ratings graph over time, you should see step changes. This would be an indication of rating manipulation, both on the positive and negative side. Many years ago I helped the seller who was the target of the fraudulent 1-star scheme prove to Amazon they were being attacked. The attack also consisted of said mob clicking on this seller's ads very early in the morning to exhaust the daily budget. So, bad reviews and your ads don't show-up when people are shopping. They got destroyed within a couple of months. Graphing a bunch of this activity clearly showed rate-of-change discrepancies. It took about six months for Amazon to finally admit this happened. They had to refund over $100K in advertising that had been consumed through fraudulent clicks. Amazon would no reveal who did this (they claimed they knew, yet refused to provide the information to enable legal action). Sadly, they did not remove any of the fraudulent ratings and reviews. Every single one of this seller's products absolutely tanked in search, going from pages 1 ~ 3 to 15+. Faced with the reality of the "mafia" behind the scenes when selling on Amazon, they felt they had no choice but to kill the business and focus on areas where open criminal manipulation and attacks were not tacitly allowed and ignored by the very ecommerce platform your business depends on. I know a bunch of people who got really hurt by Amazon through similar circumstances. To a person, most of them were people --not big business-- trying to make a go of selling product online to support their families and aim for a better life. Instead Amazon created an ecosystem that turned into a nightmare and caused most of them to lose money. Some even took out second mortgages on their homes to finance their entrepreneurial venture. Very sad.

by taeric 3y ago

stronglikedan 3y ago

My system is to just focus on the 1 & 2 star ratings, and look for trends, specifically. It takes a bit of work, since most of those reviews are just nonsense, but eventually true flaws in the product will be exposed through repetition. Due to the amount of work, I typically only reserve this method for big ticket items that would be difficult to return, such as appliances.

taeric 3y ago
Oddly, appliances are the area I would trust user reviews the least. It will be dominated by luck of experience on who will post. And with how variable manufacturing is for most, the exact same model sold on the other side of the nation is as likely to be a different beast as not. Not even considering last year's version of it. Still, I don't discard the idea here. If there are features of it that every low star mentions, I think I agree that I would trust the feature is bad. I can't think of often that has happened. Will try to pay more attention for it.

deleted 3y ago

[deleted]

rjh29 3y ago

Steam does that well, but its recommended/not recommended system is a dark pattern IMO. People tend to choose recommended even for games they don't like on the basis that someone else might like it, or it might improve in the future. Virtually every popular game has 'Overwhelmingly Positive' or similar. To get useful data you need to read the not recommended reviews in detail. A more honest system would be 'Bad', 'Average' and 'Fantastic' - then I bet you'd get a ton more 'average' reviews.

taeric 3y ago
Mostly agree, but I wouldn't call it a dark pattern. The problem is you are asking for two things. Do you want popular recommendations? Or do you want critical reviews? Those are very different and very hard to house together. This is why I mostly view the problem as being with user provided ratings. They basically fall into the same realm as all big data. Tons of noise, little signal, and easy to misinterpret.

deleted 3y ago

[deleted]

nonameiguess 3y ago

I think this highly depends on what is being rated. Non-durable consumer goods, sure, there is very little assurance that what you're getting now is even what you would have gotten ten years ago. Processes and inputs both change. A previously reputable brand might have been purchased by private equity and enshittified on purpose. On the other hand, if you're reading reviews of Moby Dick, I don't think the perceived quality or relevance of the writing is going to change much on the scale of decades. Maybe it starts to get archaic at some point, but people still get a lot out of reading Homer. If it's something like Principia Mathematica, that might honestly be timeless as long as human-like creatures exist who can read and understand math and physics.

taeric 3y ago
Fair. Books probably are a bit more evergreen on the reviews. Though, I am guessing that is highly variable on the genre? The Steam approach would probably still work rather well. No need to delete old reviews, but having recent ones can still be good to see.

_ea1k 3y ago

I've found that sorting Amazon reviews by date works pretty well. It isn't uncommon to see all 5 star reviews for a while and mostly 1 star reviews in the more recent history.

madcaptenor 3y ago
Seems like it wouldn't be too hard for Amazon to weight recent reviews more heavily.
- malfist 3y ago
  It wouldn't take too much effort for amazon to make their marketplace better in general, but they make money off of it being bad. The incentives aren't aligned, at least until reputation is hurt badly enough to cost them more profit than the short term gains they make by allowing false reviews and false advertising.
- _ea1k 3y ago
  They could, but honestly that might be risky. It might just encourage the fake reviewers to go at a slower but more steady pace to beat the system.

robomartin 3y ago

> You want me to trust ratings more, let me order them by date. Interesting idea. Having been involved with multiple third party sellers on Amazon, my extension of your thought is to have the ability to display a graph of all five rating levels over time. This makes rating manipulation visible at both ends of the scale. I know of people who, for example, sent out free samples of their product to a thousand people or more with a suggestion to post a four or five star review (after which they would get more goodies for free). I also know of sellers who were attacked by either bots or a mob (in one case a competitor hired a team outside the US) who post hundreds of 1-star ratings and negative reviews. If you could see a ratings graph over time, you should see step changes. This would be an indication of rating manipulation, both on the positive and negative side. Many years ago I helped the seller who was the target of the fraudulent 1-star scheme prove to Amazon they were being attacked. The attack also consisted of said mob clicking on this seller's ads very early in the morning to exhaust the daily budget. So, bad reviews and your ads don't show-up when people are shopping. They got destroyed within a couple of months. Graphing a bunch of this activity clearly showed rate-of-change discrepancies. It took about six months for Amazon to finally admit this happened. They had to refund over $100K in advertising that had been consumed through fraudulent clicks. Amazon would no reveal who did this (they claimed they knew, yet refused to provide the information to enable legal action). Sadly, they did not remove any of the fraudulent ratings and reviews. Every single one of this seller's products absolutely tanked in search, going from pages 1 ~ 3 to 15+. Faced with the reality of the "mafia" behind the scenes when selling on Amazon, they felt they had no choice but to kill the business and focus on areas where open criminal manipulation and attacks were not tacitly allowed and ignored by the very ecommerce platform your business depends on. I know a bunch of people who got really hurt by Amazon through similar circumstances. To a person, most of them were people --not big business-- trying to make a go of selling product online to support their families and aim for a better life. Instead Amazon created an ecosystem that turned into a nightmare and caused most of them to lose money. Some even took out second mortgages on their homes to finance their entrepreneurial venture. Very sad.