5 ms·

Surely their advanced machine learning would see such an obvious pattern?

by m3nu 7y ago

doctorpangloss 7y ago
There are more fake reviews than real ones. Due to this inversion, it is practically intractable for Amazon to detect fake reviews in a fully automated way. They must, at some point, be human moderated.
- pigscantfly 7y ago
  The class imbalance might make the modeling slightly more difficult but certainly not 'practically intractable.'
- harshreality 7y ago
  Is it so difficult? Amazon already has the number of helpful vs unhelpful votes for each review. They could start with that voting data, and then use metadata (geoip, account usage/posting timing, client fingerprinting) to create quality metrics for each user. They could even have separate trustworthiness ratings per product category (a good book reviewer might not be so good at rating appliances). From there, they'd have a rough idea of which reviews are real and fake, and they could start training a NN on the text. Figuring out when sellers are complicit, and when it's competitor sabotage, is more difficult, but Amazon has a ton of data about sellers and reviewers and buyers and the social/commercial graph connecting them. Amazon doesn't appear to be working on solving simpler problems, such as incorrect product/book comingling. Therefore, I have trouble giving them the benefit of the doubt that they're working on hard problems like addressing fraudulent ratings/sellers.
  - dfraser992 7y ago
    Good plan. See my previous comment; I did my MSc on classifying fake reviews. The behavioral / non-text features are far more useful than the text, based on all the research. There is no technical reason Amazon could not capture all the signals, except for the scale of the effort. So I'd say someone has decided it is not worth the effort/cost given the likelihood it won't improve sales that much. As for NNs, .... an ensemble of classifiers would be best (but of course I'd say that).
- jammygit 7y ago
  "More fake than real" Source?
- C1sc0cat 7y ago
  Why Google detects low quality sites in its search algo's.
- topicseed 7y ago
  True, but you take some very strong signals to classify a review as (most likely) fake de facto. Such signals could be "reviewed without purchase", "user has over 1,000 reviews", etc.
  - xnyan 7y ago
    Reviewed without purchase is a feature not a bug, according to amazon. People don't buy stuff that's not reviewed so they want to catch people who may have gotten it elsewhere. It's a sign amazon is not playing in good faith.
- dfraser992 7y ago
  I did a MSc in ML and my dissertation was on classification of fake reviews: https://douglas-fraser.com/FakeReviews/index.html https://douglas-fraser.com/FakeReviews/index.html Some years ago, there were some studies trying to estimate the percentage of fake reviews. It came out to 2% to 6%. I'm sure things have increased since then, it does look quite bad given all the anecdotes I have come across. But unless Amazon has thrown away the bulk of the 140 million or so reviews they have between 1995-2014, I doubt there really are more fake reviews than real ones now, in total. It would be interesting to do a follow up study to estimate the percentage, given all the blatant issues (such as reviews for other products, etc) Amazon is obviously ignoring. Given what I know, it shouldn't be that hard to process the obviously suspect things but the important question is always: how much profit is there for Amazon to do all that work, given it probably won't increase sales/revenue that much? It is not in Bezos' interest. The easiest thing would be to block off all of China's IP addresses....
  - qquark 7y ago
    In case you didn't notice yet, it seems like the certificate of your website is expired (2019-04-10).

Surely their advanced machine learning would see such an obvious pattern?

by m3nu 7y ago

Surely their advanced machine learning would see such an obvious pattern?

There are more fake reviews than real ones. Due to this inversion, it is practically intractable for Amazon to detect fake reviews in a fully automated way. They must, at some point, be human moderated.

pigscantfly 7y ago
The class imbalance might make the modeling slightly more difficult but certainly not 'practically intractable.'
harshreality 7y ago
Is it so difficult? Amazon already has the number of helpful vs unhelpful votes for each review. They could start with that voting data, and then use metadata (geoip, account usage/posting timing, client fingerprinting) to create quality metrics for each user. They could even have separate trustworthiness ratings per product category (a good book reviewer might not be so good at rating appliances). From there, they'd have a rough idea of which reviews are real and fake, and they could start training a NN on the text. Figuring out when sellers are complicit, and when it's competitor sabotage, is more difficult, but Amazon has a ton of data about sellers and reviewers and buyers and the social/commercial graph connecting them. Amazon doesn't appear to be working on solving simpler problems, such as incorrect product/book comingling. Therefore, I have trouble giving them the benefit of the doubt that they're working on hard problems like addressing fraudulent ratings/sellers.
- dfraser992 7y ago
  Good plan. See my previous comment; I did my MSc on classifying fake reviews. The behavioral / non-text features are far more useful than the text, based on all the research. There is no technical reason Amazon could not capture all the signals, except for the scale of the effort. So I'd say someone has decided it is not worth the effort/cost given the likelihood it won't improve sales that much. As for NNs, .... an ensemble of classifiers would be best (but of course I'd say that).
jammygit 7y ago
"More fake than real" Source?
C1sc0cat 7y ago
Why Google detects low quality sites in its search algo's.
topicseed 7y ago
True, but you take some very strong signals to classify a review as (most likely) fake de facto. Such signals could be "reviewed without purchase", "user has over 1,000 reviews", etc.
- xnyan 7y ago
  Reviewed without purchase is a feature not a bug, according to amazon. People don't buy stuff that's not reviewed so they want to catch people who may have gotten it elsewhere. It's a sign amazon is not playing in good faith.
dfraser992 7y ago
I did a MSc in ML and my dissertation was on classification of fake reviews: https://douglas-fraser.com/FakeReviews/index.html https://douglas-fraser.com/FakeReviews/index.html Some years ago, there were some studies trying to estimate the percentage of fake reviews. It came out to 2% to 6%. I'm sure things have increased since then, it does look quite bad given all the anecdotes I have come across. But unless Amazon has thrown away the bulk of the 140 million or so reviews they have between 1995-2014, I doubt there really are more fake reviews than real ones now, in total. It would be interesting to do a follow up study to estimate the percentage, given all the blatant issues (such as reviews for other products, etc) Amazon is obviously ignoring. Given what I know, it shouldn't be that hard to process the obviously suspect things but the important question is always: how much profit is there for Amazon to do all that work, given it probably won't increase sales/revenue that much? It is not in Bezos' interest. The easiest thing would be to block off all of China's IP addresses....
- qquark 7y ago
  In case you didn't notice yet, it seems like the certificate of your website is expired (2019-04-10).