6 ms·

https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law "When a measure becomes a target, it ceases to be a good measure." I

https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law "When a measure becomes a target, it ceases to be a good measure." I don't know a way around this with Amazon but it is getting increasingly annoying using a site where everything is 4-5 stars with hundreds or thousands of reviews and you need incredible intuition to tell the fake reviews from the real ones.

Judgmentality 9y ago
https://www.fakespot.com/ https://www.fakespot.com/ It's not perfect, but it's something.
- BadCookie 9y ago
  You'd think that Amazon would have some technology similar to FakeSpot's by now, but I'm not convinced that they do. I bought a product that had the special "Amazon's Choice" label on it that turned out to be a dud. I looked up the product on FakeSpot just now, and FakeSpot (correctly, I believe) identifies the reviews on that product to be unreliable (grade level "C"). Just a single anecdote, but it's clear to me that there's a ton of room for improvement on Amazon's part. Another strange thing about the "Amazon's Choice" label is this: I've seen a product get that label in part due to a low number of returns ... but the seller doesn't accept returns! That should be a con, not a pro. As you can probably tell, I have not been happy with the results of following Amazon's product choice advice.
  - save_ferris 9y ago
    I could see why they don't jump into the review verification side though. If they acquired one of these apps or built their own and another fake review comparison engine outperformed Amazon's, they'd have to deal with the backlash of why theirs isn't good enough. It feels a lot like Consumer Reports to me, where the reviews of products are handled separately from the vendor.
  - thisisit 9y ago
    Fakespot has lower negative impact if it mis-identifies a genuine product as grade level "C". But if Amazon does that you will certainly see people publishing stories about how their genuine product has been mis-identified by the giant called Amazon. For example today's submission: https://bardsandsages.com/juliedawson/2018/05/06/amazon-froze-my-account-and-i-still-dont-know-why/ https://bardsandsages.com/juliedawson/2018/05/06/amazon-froz... It will be like de-monization (of YT videos) all over again.
  - rolandog 9y ago
    I would imagine any sort of product review classification system would be exploitable for abuse. Anecdotal evidence: when my mother and her team were running for student council (in college), the opposing team introduced blatantly fake votes (photocopied) in favor of her team; my mother's team got disqualified for cheating, and the opposing team won. So I could certainly picture a way in which a mischievous seller could submit fake-looking reviews for another product and get it taken down.
    - heavenlyblue 9y ago
      There’s nothing better than the support from the college administration of such behaviour by directly acting upon it. Some of these people now sit somewhere higher up.
  - vanadium 9y ago
    Plenty of the "Amazon's Choice" flags are...a little deceptive. Maybe not deceptive in a conventional sense, but say you see a product in the search results on Amazon and one has that flag. You click on it, because hey. Then you notice the keywords attached to "Amazon's Choice". Oftentimes, they're strange and esoteric. Oftentimes, that flag is put on what appears to be a random off-brand item sitting right next to a much, much better product more deserving of that "Choice" insignia.
    - vertexFarm 9y ago
      From what I've seen I'm guessing the amazon's choice tag is arbitrary and generated by some algorithm that simply matches up common search terms to the most-bought item. When it's something simple and common like "wireless mouse" it's probably alright. When it's a bit more esoteric like "male displayport socket soldered connector" or some other hopeless attempt to optimize search results and filter out noise, you get strange things.
  - stordoff 9y ago
    Part of the problem for Amazon introducing it is that it could be seen as vouching for reviews that were not detected as fake, even though they may still be fake. Unless the false negative rate is sufficiently low, it may give false confidence in products. > but the seller doesn't accept returns! That should be a con, not a pro. US? In the UK, the seller has to accept returns, so that removes one variable (and if the product is bad, you can just send it back).
- psychometry 9y ago
  I'm pretty sure Fakespot has no idea what they're doing. They provide very little detail on methodology and I've found a lot of their letter grades highly suspect. It's easy enough to assess their accuracy by looking at some items that have few enough reviews that you can tell yourself whether they're real or not. More often than not, I've found that Fakespot's grades are terrible.
  - zrth 9y ago
    I wonder if it would make sense for vendors to post obvious fake reviews on their competitors product pages to discredit these.
DarkTree 9y ago
For sure, this is a prime example of Goodhart's law in action. Before Amazon came along, I rarely depended on reading reviews before buying a product. Yes, there were definitely sites where reviews helped persuade a purchase, for example Newegg.com, but at that time it wasn't required that I scour a list of reviews before buying a product, it was more of a scavenger hunt around the internet for personal reviews/blogs, or just not at all. Then Amazon came along, and it transformed the way I buy products. But now, you're right. It has gotten so difficult to trust any review and I often find myself ping-ponging around competing products trying to intuit what reviews are the least fake. Good reviews have simply become a target that don't necessarily signal a good product.
- animal531 9y ago
  Sounds like a business opportunity for someone to introduce the meta review, which will review and rank all reviews for a given product and/or search.
Kluny 9y ago
I think this is the tragedy of the commons in action. In this the case, the "common" is our collective willingness to post helpful information on websites for other customers. From my point of view, I'm adding value to the merchant and getting nothing in return. I used to get the fuzzy warm feeling of contributing to the collective good of humanity, but now that it's so commoditized I'm not willing to do it anymore. There are two solutions that I know of; the socialist one and the capitalist one. The socialist solution is some kind of collectively-funded watchdog organization that provides useful review information. The capitalist solution is an organization that straight up charges for reviews, ie, Consumer Reports. I'm sure someone's working on their startup about it now.
- aninhumer 9y ago
  >The socialist solution is some kind of collectively-funded watchdog organization that provides useful review information. That's more like the liberal social democrat solution. The Socialist solution is to abolish exchange based production of goods so there's no longer an incentive to lie to increase sales of poor quality products.
  - Kluny 9y ago
    That sounds more like the communist option to me.
CamelCaseName 9y ago
> it is getting increasingly annoying using a site where everything is 4-5 stars with hundreds or thousands of reviews and you need incredible intuition to tell the fake reviews from the real ones. While it's definitely a problem, there are WAY more 1-3 star listings than you know, but you'll never see them because the sellers stop selling them or their sales slow down and are dropped from the rankings.
- tracker1 9y ago
  I think part of the problem, is that there really needs to be a "seller verification" process, similar to an EV cert for sellers, that's easier for users to filter. It used to be enough to filter "Fullfilled By Amazon" .. hell, now I mostly click the "Seller: Amazon" option, but it feels like Amazon has reduced their own inventory in favor of 3rd party sellers.
  - CamelCaseName 9y ago
    The reason why Amazon reduced their vendor program is because they were getting scammed. Here's an example: Vendor sells Amazon 500 shirts at $4/unit, retails at $10. Once in stock, vendor sells the shirts very quickly. They do this by either putting the shirt on sale for pennies, or by using hacked accounts to buy the shirts (or by paying people to buy the shirts). What happens when Amazon sees that the shirts they ordered sold out in 3 days, and are going to take 30 days to replenish? They buy more, and although the vendor may have initially been down the $4 * 500, when Amazon places that reorder for 5,000+ units, the vendor is up substantially and bails, leaving Amazon with a ton of inventory.
vertexFarm 9y ago
One (partial) solution would be to simply eliminate reviews. They're compromised. They aren't worth anything anymore and are just taking up bandwidth and bothering users. Not to mention fostering a weird little black market. Just make people research what they're buying elsewhere--which will probably also get swamped with fraud--or just let customers roll the dice without the false sense of security. They're already doing it, but with a bunch of fake 5-stars instead of a bunch of unreviewed items.

Judgmentality 9y ago

https://www.fakespot.com/ https://www.fakespot.com/ It's not perfect, but it's something.

BadCookie 9y ago
You'd think that Amazon would have some technology similar to FakeSpot's by now, but I'm not convinced that they do. I bought a product that had the special "Amazon's Choice" label on it that turned out to be a dud. I looked up the product on FakeSpot just now, and FakeSpot (correctly, I believe) identifies the reviews on that product to be unreliable (grade level "C"). Just a single anecdote, but it's clear to me that there's a ton of room for improvement on Amazon's part. Another strange thing about the "Amazon's Choice" label is this: I've seen a product get that label in part due to a low number of returns ... but the seller doesn't accept returns! That should be a con, not a pro. As you can probably tell, I have not been happy with the results of following Amazon's product choice advice.
- save_ferris 9y ago
  I could see why they don't jump into the review verification side though. If they acquired one of these apps or built their own and another fake review comparison engine outperformed Amazon's, they'd have to deal with the backlash of why theirs isn't good enough. It feels a lot like Consumer Reports to me, where the reviews of products are handled separately from the vendor.
- thisisit 9y ago
  Fakespot has lower negative impact if it mis-identifies a genuine product as grade level "C". But if Amazon does that you will certainly see people publishing stories about how their genuine product has been mis-identified by the giant called Amazon. For example today's submission: https://bardsandsages.com/juliedawson/2018/05/06/amazon-froze-my-account-and-i-still-dont-know-why/ https://bardsandsages.com/juliedawson/2018/05/06/amazon-froz... It will be like de-monization (of YT videos) all over again.
- rolandog 9y ago
  I would imagine any sort of product review classification system would be exploitable for abuse. Anecdotal evidence: when my mother and her team were running for student council (in college), the opposing team introduced blatantly fake votes (photocopied) in favor of her team; my mother's team got disqualified for cheating, and the opposing team won. So I could certainly picture a way in which a mischievous seller could submit fake-looking reviews for another product and get it taken down.
  - heavenlyblue 9y ago
    There’s nothing better than the support from the college administration of such behaviour by directly acting upon it. Some of these people now sit somewhere higher up.
- vanadium 9y ago
  Plenty of the "Amazon's Choice" flags are...a little deceptive. Maybe not deceptive in a conventional sense, but say you see a product in the search results on Amazon and one has that flag. You click on it, because hey. Then you notice the keywords attached to "Amazon's Choice". Oftentimes, they're strange and esoteric. Oftentimes, that flag is put on what appears to be a random off-brand item sitting right next to a much, much better product more deserving of that "Choice" insignia.
  - vertexFarm 9y ago
    From what I've seen I'm guessing the amazon's choice tag is arbitrary and generated by some algorithm that simply matches up common search terms to the most-bought item. When it's something simple and common like "wireless mouse" it's probably alright. When it's a bit more esoteric like "male displayport socket soldered connector" or some other hopeless attempt to optimize search results and filter out noise, you get strange things.
- stordoff 9y ago
  Part of the problem for Amazon introducing it is that it could be seen as vouching for reviews that were not detected as fake, even though they may still be fake. Unless the false negative rate is sufficiently low, it may give false confidence in products. > but the seller doesn't accept returns! That should be a con, not a pro. US? In the UK, the seller has to accept returns, so that removes one variable (and if the product is bad, you can just send it back).
psychometry 9y ago
I'm pretty sure Fakespot has no idea what they're doing. They provide very little detail on methodology and I've found a lot of their letter grades highly suspect. It's easy enough to assess their accuracy by looking at some items that have few enough reviews that you can tell yourself whether they're real or not. More often than not, I've found that Fakespot's grades are terrible.
- zrth 9y ago
  I wonder if it would make sense for vendors to post obvious fake reviews on their competitors product pages to discredit these.

DarkTree 9y ago

For sure, this is a prime example of Goodhart's law in action. Before Amazon came along, I rarely depended on reading reviews before buying a product. Yes, there were definitely sites where reviews helped persuade a purchase, for example Newegg.com, but at that time it wasn't required that I scour a list of reviews before buying a product, it was more of a scavenger hunt around the internet for personal reviews/blogs, or just not at all. Then Amazon came along, and it transformed the way I buy products. But now, you're right. It has gotten so difficult to trust any review and I often find myself ping-ponging around competing products trying to intuit what reviews are the least fake. Good reviews have simply become a target that don't necessarily signal a good product.

animal531 9y ago
Sounds like a business opportunity for someone to introduce the meta review, which will review and rank all reviews for a given product and/or search.

Kluny 9y ago

I think this is the tragedy of the commons in action. In this the case, the "common" is our collective willingness to post helpful information on websites for other customers. From my point of view, I'm adding value to the merchant and getting nothing in return. I used to get the fuzzy warm feeling of contributing to the collective good of humanity, but now that it's so commoditized I'm not willing to do it anymore. There are two solutions that I know of; the socialist one and the capitalist one. The socialist solution is some kind of collectively-funded watchdog organization that provides useful review information. The capitalist solution is an organization that straight up charges for reviews, ie, Consumer Reports. I'm sure someone's working on their startup about it now.

aninhumer 9y ago
>The socialist solution is some kind of collectively-funded watchdog organization that provides useful review information. That's more like the liberal social democrat solution. The Socialist solution is to abolish exchange based production of goods so there's no longer an incentive to lie to increase sales of poor quality products.
- Kluny 9y ago
  That sounds more like the communist option to me.

CamelCaseName 9y ago

> it is getting increasingly annoying using a site where everything is 4-5 stars with hundreds or thousands of reviews and you need incredible intuition to tell the fake reviews from the real ones. While it's definitely a problem, there are WAY more 1-3 star listings than you know, but you'll never see them because the sellers stop selling them or their sales slow down and are dropped from the rankings.

tracker1 9y ago
I think part of the problem, is that there really needs to be a "seller verification" process, similar to an EV cert for sellers, that's easier for users to filter. It used to be enough to filter "Fullfilled By Amazon" .. hell, now I mostly click the "Seller: Amazon" option, but it feels like Amazon has reduced their own inventory in favor of 3rd party sellers.
- CamelCaseName 9y ago
  The reason why Amazon reduced their vendor program is because they were getting scammed. Here's an example: Vendor sells Amazon 500 shirts at $4/unit, retails at $10. Once in stock, vendor sells the shirts very quickly. They do this by either putting the shirt on sale for pennies, or by using hacked accounts to buy the shirts (or by paying people to buy the shirts). What happens when Amazon sees that the shirts they ordered sold out in 3 days, and are going to take 30 days to replenish? They buy more, and although the vendor may have initially been down the $4 * 500, when Amazon places that reorder for 5,000+ units, the vendor is up substantially and bails, leaving Amazon with a ton of inventory.

vertexFarm 9y ago

One (partial) solution would be to simply eliminate reviews. They're compromised. They aren't worth anything anymore and are just taking up bandwidth and bothering users. Not to mention fostering a weird little black market. Just make people research what they're buying elsewhere--which will probably also get swamped with fraud--or just let customers roll the dice without the false sense of security. They're already doing it, but with a bunch of fake 5-stars instead of a bunch of unreviewed items.