6 ms·

The intensity and speed this thread rose up on HN illustrates a very important principle. People are more likely to leave a review/comment/upvote if something i

The intensity and speed this thread rose up on HN illustrates a very important principle. People are more likely to leave a review/comment/upvote if something is not working for them. Understanding that is useful when comparing products using reviews.

btilly 13y ago
This is a very important principle. For a time, Target used Amazon's website code. Here is what they found. People leave reviews of books that they like. People only bother reviewing microwave machines when they didn't work. This meant that Amazon's code didn't work out as well for Target as it did for Amazon...
- IgorPartola 13y ago
  The whole online review thing is thoroughly broken. I am not even talking about people leaving bad reviews for things like "the seller didn't ship it on time", etc. I am talking about the fact that most people that leave a review have nothing to compare the product to. For example, try shopping for a wireless router. You'll find lots of people either praising or bashing it but having no basis for comparison. Star ratings are completely screwed because lots of people give both one and five star ratings based on completely insane criteria. Long written reviews are naturally most informative, but you have to remember that they are also extremely subjective. On the other hand, you can't trust expert reviews either. The experts spend too little time with each product to form an actual opinion. They often form their opinions based on some very shallow factors. A canonical example of this is mobile phone reviews by sites like Engadget or Gizmodo. They review so many phones that they can spend only a few days with each. Edit: one of the worst things for this is reviews of hard drives. While there certainly are hard numbers to go off, there are also lots of opinions: "Well I've owned 6 Seagate drives over the past 4 years, and they all ended up dead. Seagate products are crap!" That does not mean that all Seagate drives are crap, maybe just laptop ones. Or maybe your controller is bad and you keep attaching good drives to it. Or maybe you keep sticking these drives into a case with poor ventilation. On top of that your sample size is too small.
  - sourc3 13y ago
    If you want to fix this with me drop me a note. I have an idea, a design and a viable business model.
  - matznerd 13y ago
    Are there offline reviews that are better? Do you trust the information of a salesperson better? I think there are definitely problems with a selection bias of who leaves reviews, but sites like Amazon have implemented a number of features to improve upon them like rating distributions (how many 5, 4, 3 ,2, and 1s), the vine program, reviewer rankings, verified purchases, large volumes etc.
    - ericd 13y ago
      Consumer Reports is actually pretty damn good for most things, especially in terms of reliability ratings. Good reviews is one of the things I'm happy to pay for, since the web doesn't seem to have spawned a good free alternative.
      - TrainedMonkey 13y ago
        Ah, that convinced me to actually buy membership. Thanks for spreading the word.
        ericd 13y ago
        Yeah, it's really a great publication. I really enjoy that they've taken an ideological stance to never take advertising so that they never have a conflict of interest, in stark contrast to almost every review site on the internet. When testing cars, they have plainclothes buyers to buy the cars, as well (partly so that they're not subject to keeping companies happy to keep the review samples flowing). They seem to also have pretty involved testing methodologies. All of this takes money, and I imagine that the rise of the internet hasn't been kind to their cashflow, so I'm happy to chip in to keep this sort of independent testing organization alive.
      - 13y ago
- gt384u 13y ago
  > For a time, Target used Amazon's website code. I remember getting the email saying that we no longer had to special case Target stuff in the retail platform. 'Twas a joyous day all around.
pessimizer 13y ago
I think more that people are more likely to communicate when things change rather than when they don't. "This is your pilot speaking. The engines are currently not on fire and are still attached to the plane. That is all."
hnriot 13y ago
isn't that obvious? it's because being down is the exception, it would be pointless commenting that gmail is up since that's the ordinary state.
baddox 13y ago
I wonder if you could utilize this information. Perhaps negative reviews could get more weight if they're from people who have also left positive reviews.
flippyhead 13y ago
I think half the reason people upvote stories about big important sites like gmail being down is because it makes them feel better about any downtime their own sites have had.
- Zancarius 13y ago
  I must be terribly naive or hopelessly optimistic. The immediate reason I thought of that would explain why stories like this are upvoted is for a matter of visibility, and in a distant second place, to head off questions like "Is it broken for you, too?" Judging by a cursory review of the comment here, I suspect that reasoning is fairly common. It strikes me as awfully pessimistic to immediately assume someone's clicking their mouse thinking "Haha, Gmail is down! Take THAT Google! >upvote<" when the result is very real service disruption for a lot of people.
  - flippyhead 13y ago
    I'm sure it's a mix of reasons. Like I said, my guess is half. Also, it wan't an immediate assumption, I've thought about it for a long time. Lastly, I've not suggested people are glad gmail is down or that people are harmed. That hadn't occurred to me; but now that you mention it, you're right, it probably does happen occasionally.
bborud 13y ago
Oh. My first thought was "Ouch, we're too dependent on Gmail", but sure :)
axx 13y ago
True, everytime we change something on our product, only the people that have something to complain about, say something. Every productguy/developer should know this. :)
bostik 13y ago
> People are more likely to leave a review/comment/upvote if something is not working for them. Also known as the 3/11 rule. On average people tell 3 others when they have had positive experience. And on average they tell 11 others when they have a negative experience. [proper citation appreciated] Add the effect of internet, and multiply by Google effect. More recent posts are given higher relevance, and of course more frequent occurrences are given higher relevance. News agencies and publishers have known this for ages - after all, there's no news like bad news.

btilly 13y ago

This is a very important principle. For a time, Target used Amazon's website code. Here is what they found. People leave reviews of books that they like. People only bother reviewing microwave machines when they didn't work. This meant that Amazon's code didn't work out as well for Target as it did for Amazon...

IgorPartola 13y ago
The whole online review thing is thoroughly broken. I am not even talking about people leaving bad reviews for things like "the seller didn't ship it on time", etc. I am talking about the fact that most people that leave a review have nothing to compare the product to. For example, try shopping for a wireless router. You'll find lots of people either praising or bashing it but having no basis for comparison. Star ratings are completely screwed because lots of people give both one and five star ratings based on completely insane criteria. Long written reviews are naturally most informative, but you have to remember that they are also extremely subjective. On the other hand, you can't trust expert reviews either. The experts spend too little time with each product to form an actual opinion. They often form their opinions based on some very shallow factors. A canonical example of this is mobile phone reviews by sites like Engadget or Gizmodo. They review so many phones that they can spend only a few days with each. Edit: one of the worst things for this is reviews of hard drives. While there certainly are hard numbers to go off, there are also lots of opinions: "Well I've owned 6 Seagate drives over the past 4 years, and they all ended up dead. Seagate products are crap!" That does not mean that all Seagate drives are crap, maybe just laptop ones. Or maybe your controller is bad and you keep attaching good drives to it. Or maybe you keep sticking these drives into a case with poor ventilation. On top of that your sample size is too small.
- sourc3 13y ago
  If you want to fix this with me drop me a note. I have an idea, a design and a viable business model.
- matznerd 13y ago
  Are there offline reviews that are better? Do you trust the information of a salesperson better? I think there are definitely problems with a selection bias of who leaves reviews, but sites like Amazon have implemented a number of features to improve upon them like rating distributions (how many 5, 4, 3 ,2, and 1s), the vine program, reviewer rankings, verified purchases, large volumes etc.
  - ericd 13y ago
    Consumer Reports is actually pretty damn good for most things, especially in terms of reliability ratings. Good reviews is one of the things I'm happy to pay for, since the web doesn't seem to have spawned a good free alternative.
    - TrainedMonkey 13y ago
      Ah, that convinced me to actually buy membership. Thanks for spreading the word.
      - ericd 13y ago
        Yeah, it's really a great publication. I really enjoy that they've taken an ideological stance to never take advertising so that they never have a conflict of interest, in stark contrast to almost every review site on the internet. When testing cars, they have plainclothes buyers to buy the cars, as well (partly so that they're not subject to keeping companies happy to keep the review samples flowing). They seem to also have pretty involved testing methodologies. All of this takes money, and I imagine that the rise of the internet hasn't been kind to their cashflow, so I'm happy to chip in to keep this sort of independent testing organization alive.
    - 13y ago
gt384u 13y ago
> For a time, Target used Amazon's website code. I remember getting the email saying that we no longer had to special case Target stuff in the retail platform. 'Twas a joyous day all around.

pessimizer 13y ago

I think more that people are more likely to communicate when things change rather than when they don't. "This is your pilot speaking. The engines are currently not on fire and are still attached to the plane. That is all."

hnriot 13y ago

isn't that obvious? it's because being down is the exception, it would be pointless commenting that gmail is up since that's the ordinary state.

baddox 13y ago

I wonder if you could utilize this information. Perhaps negative reviews could get more weight if they're from people who have also left positive reviews.

flippyhead 13y ago

I think half the reason people upvote stories about big important sites like gmail being down is because it makes them feel better about any downtime their own sites have had.

Zancarius 13y ago
I must be terribly naive or hopelessly optimistic. The immediate reason I thought of that would explain why stories like this are upvoted is for a matter of visibility, and in a distant second place, to head off questions like "Is it broken for you, too?" Judging by a cursory review of the comment here, I suspect that reasoning is fairly common. It strikes me as awfully pessimistic to immediately assume someone's clicking their mouse thinking "Haha, Gmail is down! Take THAT Google! >upvote<" when the result is very real service disruption for a lot of people.
- flippyhead 13y ago
  I'm sure it's a mix of reasons. Like I said, my guess is half. Also, it wan't an immediate assumption, I've thought about it for a long time. Lastly, I've not suggested people are glad gmail is down or that people are harmed. That hadn't occurred to me; but now that you mention it, you're right, it probably does happen occasionally.

bborud 13y ago

Oh. My first thought was "Ouch, we're too dependent on Gmail", but sure :)

axx 13y ago

True, everytime we change something on our product, only the people that have something to complain about, say something. Every productguy/developer should know this. :)

bostik 13y ago

> People are more likely to leave a review/comment/upvote if something is not working for them. Also known as the 3/11 rule. On average people tell 3 others when they have had positive experience. And on average they tell 11 others when they have a negative experience. [proper citation appreciated] Add the effect of internet, and multiply by Google effect. More recent posts are given higher relevance, and of course more frequent occurrences are given higher relevance. News agencies and publishers have known this for ages - after all, there's no news like bad news.