4 ms·

Our critics appear to be less educated than our fans

vishaldpatel 12y ago
Great article! Really makes you think about the 2 people that give an otherwise 5-star item a 1 star, bringing it's rating down to 4.5 stars. The thing is, I tend to read over those reviews because of all the bots and hired people / friends that are padding the 5-star reviews.
thejteam 12y ago
One thing I should point out is that the readability formulas that they used were not designed for short form text like reviews. Most of them were designed for text that is at least 100 words.
- drawkbox 12y ago
  Good point, maybe bad reviews should have a length requirement. Might be more constructive or get rid of the 1-star reviews like "didn't work", "couldn't download" for apps/games as well.
mc32 12y ago
If the statistics significance is small due to a paucity of poor reviews, why not apply the test to other books on Amazon, or other products altogether?
- bduerst 12y ago
  Good idea, but they're assuming their product is good. What if other products are bad? The 'intelligent' reviewers may write one star reviews on other books that are legitimately bad books. If anything, you could use this API to calculate a "True" amazon book review, in which you could identify the reading level of the reviewers and weight their # star review accordingly. I might give a crack at this later because I spend so much time trying to decipher Amazon's reviews regardless.
  - mc32 12y ago
    A few things come to mind. Are the reviews of more proficient (educated, as determined by their algo) writers more accurate overall? That's to say are their bad reviews of products as accurate as their good reviews, if we track people across products. Or do good writers avoid writing bad reviews? Are the reviews of poor writers who write good reviews any less reliable? And can they apply this to Netflix?
long 12y ago
This reminded me of the style of the Cards Against Humanity folks, i.e., irreverent and hilarious
nathan_long 12y ago
In case you're taking this seriously, consider other possible explanations. Maybe more eloquent people are less likely to write negative reviews in general, not just for this book. Maybe feeling negative makes a person less eloquent. Maybe this book was marketed to a group of people who are generally educated but gullible when it comes to the subject matter. It could be that their critics are idiots, but this isn't much evidence. (And I'm pretty sure they're joking.)
- bduerst 12y ago
  A way to test this hypothesis would be to look at the range of star reviews that the reviewer has given, and see if "higher education" reviewers do in fact give more positive, optimistic reviews on a whole.
belorn 12y ago
What is the incentives to write a long well thought out and interesting negative review on amazon? I would guess that while criticism tend to end up as blog posts, positive reviews end up on amazon because people want to share a positive experience with others.
rogerbinns 12y ago
If you are giving a one star review then you are likely in a situation where the item has wasted your time, and frustrated/annoyed/disappointed you. That is not conducive to good writing. It would be better to check the reading level of reviews given by the same person against other products, as well as the scores to get a clearer view of them.
deleted 12y ago
[deleted]
SilasX 12y ago
They seem to be proud of how 4/5 star reviewer are more intelligent[1] than 1-3. They de-emphasize how 2-star reviewers are more intelligent than 3-star reviewers, or how their readership will be skewed toward those who already liked their work. [1] to the extent measured by this proxy
dang 12y ago
We changed the over-the-top article title to the slightly less baity chart caption.

Great article! Really makes you think about the 2 people that give an otherwise 5-star item a 1 star, bringing it's rating down to 4.5 stars. The thing is, I tend to read over those reviews because of all the bots and hired people / friends that are padding the 5-star reviews.

thejteam 12y ago

One thing I should point out is that the readability formulas that they used were not designed for short form text like reviews. Most of them were designed for text that is at least 100 words.

drawkbox 12y ago
Good point, maybe bad reviews should have a length requirement. Might be more constructive or get rid of the 1-star reviews like "didn't work", "couldn't download" for apps/games as well.

mc32 12y ago

If the statistics significance is small due to a paucity of poor reviews, why not apply the test to other books on Amazon, or other products altogether?

bduerst 12y ago
Good idea, but they're assuming their product is good. What if other products are bad? The 'intelligent' reviewers may write one star reviews on other books that are legitimately bad books. If anything, you could use this API to calculate a "True" amazon book review, in which you could identify the reading level of the reviewers and weight their # star review accordingly. I might give a crack at this later because I spend so much time trying to decipher Amazon's reviews regardless.
- mc32 12y ago
  A few things come to mind. Are the reviews of more proficient (educated, as determined by their algo) writers more accurate overall? That's to say are their bad reviews of products as accurate as their good reviews, if we track people across products. Or do good writers avoid writing bad reviews? Are the reviews of poor writers who write good reviews any less reliable? And can they apply this to Netflix?

long 12y ago

This reminded me of the style of the Cards Against Humanity folks, i.e., irreverent and hilarious

nathan_long 12y ago

In case you're taking this seriously, consider other possible explanations. Maybe more eloquent people are less likely to write negative reviews in general, not just for this book. Maybe feeling negative makes a person less eloquent. Maybe this book was marketed to a group of people who are generally educated but gullible when it comes to the subject matter. It could be that their critics are idiots, but this isn't much evidence. (And I'm pretty sure they're joking.)

bduerst 12y ago
A way to test this hypothesis would be to look at the range of star reviews that the reviewer has given, and see if "higher education" reviewers do in fact give more positive, optimistic reviews on a whole.

belorn 12y ago

What is the incentives to write a long well thought out and interesting negative review on amazon? I would guess that while criticism tend to end up as blog posts, positive reviews end up on amazon because people want to share a positive experience with others.

rogerbinns 12y ago

If you are giving a one star review then you are likely in a situation where the item has wasted your time, and frustrated/annoyed/disappointed you. That is not conducive to good writing. It would be better to check the reading level of reviews given by the same person against other products, as well as the scores to get a clearer view of them.

deleted 12y ago

[deleted]

SilasX 12y ago

They seem to be proud of how 4/5 star reviewer are more intelligent[1] than 1-3. They de-emphasize how 2-star reviewers are more intelligent than 3-star reviewers, or how their readership will be skewed toward those who already liked their work. [1] to the extent measured by this proxy

dang 12y ago

We changed the over-the-top article title to the slightly less baity chart caption.