5 ms·

https://douglas-fraser.com/FakeReviews/index.html https://douglas-fraser.com/FakeReviews/index.html If anyone is interested... it is my dissertation on develop

https://douglas-fraser.com/FakeReviews/index.html https://douglas-fraser.com/FakeReviews/index.html If anyone is interested... it is my dissertation on developing ML ensembles to classify reviews as fake or not, strictly using features derived from textual analysis (deception theory, stylometry, etc). Behavioral type features, like IP address and relationships between spamming accounts are more useful, but the text still has some value. At some point, I will expand this to using actual Amazon reviews, but I had to drop that idea just to get the dissertation done on time. Several years ago, the percentage of fake reviews was estimated at 2 to 6%, so I'm sure it has increased to (gut feeling) probably like 10% these days. Is that "tiny"? A really good stat to calculate would be the percentage of products that have more than just a few fake reviews - such that the consumer is given a bad impression. That stat probably is not "tiny" and thus creates the perception Amazon reviews are useless.

tudelo 8y ago
I think the concentration of fake reviews varies a lot with the product, so for some products, they are essentially useless.
DenisM 8y ago
Are you using NLP of any sort to assist with feature extraction? Curious.
- dfraser992 8y ago
  Yes - I primarily used spaCy [0] to handle all the NLP related tasks (tokenization, etc) as well as empath [1]. The Stanford Parser was only used for constituency parsing as spaCy doesn't handle that. The different feature sets were then derived from all that data. [0]: https://spacy.io https://spacy.io [1]: https://github.com/Ejhfast/empath-client https://github.com/Ejhfast/empath-client
mmt 8y ago
>so I'm sure it has increased to (gut feeling) probably like 10% these days That's consistent with the algorithmic estimate quoted in the article: 'the ReviewMeta algorithm labeled 9.1%, or 5.3 million of the dataset's reviews, as “unnatural.”' I would say that's far from tiny, especially considering how many other reviews (at least in my own experience) have inadequate information in their text. A 3-star review that has only the words "It worked just fine but nothing special" is just as useless to me as a fake 5-star or 1-star review, whatever the fake text.

tudelo 8y ago

I think the concentration of fake reviews varies a lot with the product, so for some products, they are essentially useless.

DenisM 8y ago

Are you using NLP of any sort to assist with feature extraction? Curious.

dfraser992 8y ago
Yes - I primarily used spaCy [0] to handle all the NLP related tasks (tokenization, etc) as well as empath [1]. The Stanford Parser was only used for constituency parsing as spaCy doesn't handle that. The different feature sets were then derived from all that data. [0]: https://spacy.io https://spacy.io [1]: https://github.com/Ejhfast/empath-client https://github.com/Ejhfast/empath-client

mmt 8y ago

>so I'm sure it has increased to (gut feeling) probably like 10% these days That's consistent with the algorithmic estimate quoted in the article: 'the ReviewMeta algorithm labeled 9.1%, or 5.3 million of the dataset's reviews, as “unnatural.”' I would say that's far from tiny, especially considering how many other reviews (at least in my own experience) have inadequate information in their text. A 3-star review that has only the words "It worked just fine but nothing special" is just as useless to me as a fake 5-star or 1-star review, whatever the fake text.