4 ms·
The whole online review thing is thoroughly broken. I am not even talking about people leaving bad reviews for things like "the seller didn't ship it on time",
by IgorPartola 13y ago
The whole online review thing is thoroughly broken. I am not even talking about people leaving bad reviews for things like "the seller didn't ship it on time", etc. I am talking about the fact that most people that leave a review have nothing to compare the product to. For example, try shopping for a wireless router. You'll find lots of people either praising or bashing it but having no basis for comparison. Star ratings are completely screwed because lots of people give both one and five star ratings based on completely insane criteria. Long written reviews are naturally most informative, but you have to remember that they are also extremely subjective.
On the other hand, you can't trust expert reviews either. The experts spend too little time with each product to form an actual opinion. They often form their opinions based on some very shallow factors. A canonical example of this is mobile phone reviews by sites like Engadget or Gizmodo. They review so many phones that they can spend only a few days with each.
Edit: one of the worst things for this is reviews of hard drives. While there certainly are hard numbers to go off, there are also lots of opinions: "Well I've owned 6 Seagate drives over the past 4 years, and they all ended up dead. Seagate products are crap!" That does not mean that all Seagate drives are crap, maybe just laptop ones. Or maybe your controller is bad and you keep attaching good drives to it. Or maybe you keep sticking these drives into a case with poor ventilation. On top of that your sample size is too small.
- sourc3 13y agoIf you want to fix this with me drop me a note. I have an idea, a design and a viable business model.
- matznerd 13y agoAre there offline reviews that are better? Do you trust the information of a salesperson better? I think there are definitely problems with a selection bias of who leaves reviews, but sites like Amazon have implemented a number of features to improve upon them like rating distributions (how many 5, 4, 3 ,2, and 1s), the vine program, reviewer rankings, verified purchases, large volumes etc.
- ericd 13y agoConsumer Reports is actually pretty damn good for most things, especially in terms of reliability ratings. Good reviews is one of the things I'm happy to pay for, since the web doesn't seem to have spawned a good free alternative.
- TrainedMonkey 13y agoAh, that convinced me to actually buy membership. Thanks for spreading the word.
- ericd 13y agoYeah, it's really a great publication. I really enjoy that they've taken an ideological stance to never take advertising so that they never have a conflict of interest, in stark contrast to almost every review site on the internet. When testing cars, they have plainclothes buyers to buy the cars, as well (partly so that they're not subject to keeping companies happy to keep the review samples flowing). They seem to also have pretty involved testing methodologies. All of this takes money, and I imagine that the rise of the internet hasn't been kind to their cashflow, so I'm happy to chip in to keep this sort of independent testing organization alive.
- Nagyman 13y agoPSA: Some libraries have free access to Consumer Reports. Check with your local library to see if they have a membership.
- malandrew 13y agoFor cooking stuff, Cook's Illustrated is the reference. Are there better recipes and cooking equipment reviews out there? Definitely. But it's generally of very high quality.
- IgorPartola 13y agoI trust some reviews of some products by some people. I differentiate. For example, for car reviews I read Edmunds reviews but don't trust all of their opinions. They don't try to do anything with car reliability which really sucks, but they can't. You don't know what's what until you've owned the car for 10 years, and they are reviewing new cars. I trust my mechanic friend much more here since he can actually look under the hood and tell me if the type of car has a transmission that's prone to go bad, or an engine that's going to burn oil. It's possible that he's pulling his opinions out of his own biases, but what he says usually makes internal sense so I feel better if he tells me something is good or bad. I don't trust good restaurant reviews, and only trust bad ones if they cite specific issues, like using wrong ingredients, etc. I trust my friends and family when they say a place is good or bad much more because I know their tastes and I can ask followup questions. For electronics, I pretty much don't trust anyone with anything unless something is backed by the manufacturer. I only trust Apple products because of Apple Care. Everyone has had a bad experience with a laptop, a phone, a tablet, a mouse, a USB cable, etc. It's not worth listening to most times since it's usually a horror story from 10 years ago about a company that who's had their entire manufacturing line re-tooled since. On Amazon for a generic product, the biggest indication to me is if there are lots of reviews. That means lots of people own the product. If most of the reviews are one star, it's a crappy product. If not, it could be good or bad and I personally find very little correlation between the way I perceive a product and its Amazon reviews.
- dclara 13y ago"The whole online review thing is thoroughly broken." "Star ratings are completely screwed" "Long written reviews "...are also extremely subjective." Totally agree. Here is my analysis and resolution: http://bit.ly/LRSdkJ http://bit.ly/LRSdkJ or If you want to fix this with me drop me a note. I have an idea, a design and a viable business model. :)
- sliverstorm 13y agoThey aren't great but they aren't useless either. A customer is basically rating, "Was I satisfied?". If everyone who bought this product was 100.0% satisfied, is that not useful information, even if there is another competing product that might be better?
- Florin_Andrei 13y agoThe thing is, there is no way to ascertain the level of competence of the person writing the review. Is this someone really knowledgeable, or is it some dumb newbie? Sometimes it's easy to tell, but other times it's very hard. Then there's also the issue of reviews with an "agenda" - self-explanatory, no need for me to say more. One thing that seems to work is to see someone I know using a gadget, to actually see that gadget being used, and talk to that individual briefly about the gadget. The info I get this way seems quite a bit more trustworthy than online reviews. Of course, this could also be an artifact of the way we form opinions. Another empirical observation: on Amazon, a significant amount of negative reviews is typically an indication of a real issue. But the absence of negative reviews doesn't say much. Positive reviews do not appear to be quite as relevant-information-rich, unless there's an extremely large amount of them.
- avenger123 13y agoIt's sometimes great pleasure to see how an original topic in HN starts delving into side topics that have nothing to do with the original. I thought it was a one-off thing but seems to happen quite a bit.
- FilterJoe 13y agoAgreed that online reviews are broken. That's what drove me to co-found a company that takes a different approach to reviews, one which stresses category over individual products. Start with the premise that the person you most want to give you advice is someone who is passionate about a topic or at least has done a lot research on it. If you ask them for advice, first they'll give you an overview, which hopefully includes what attributes you should be thinking about. Then they'll recommend a small number of models to consider, based on what you seem to be interested in. If you're doing this with other people around, perhaps a few other people will pipe in with suggestions. So - for example - if you're looking for 3-4 player board games more interesting than Sorry and Monopoly - so interesting that they have a decent chance of getting your friends and/or family members more into gaming - I prepared a guide on our site that does just that: http://obviously.com/619/gateway-games http://obviously.com/619/gateway-games (Note: site styling is incomplete - we're not quite ready to launch) The essential idea is that if you narrow a topic down enough (i.e. 7200 RPM HDDs that are very quiet and can easily survive a several foot drop onto a hard floor), there's probably somebody out there who's taken a great interest in that narrow niche and can write about it very well. On the other hand, it's hard to write about the too-broad hard drive category as a whole, and it's also difficult to talk about one hard drive with little context.
- instakill 13y agoAgreed about online reviews being imperfect. However one way to counteract extreme subjectivity is context. If you know somebody in person, you'll be able to understand their online review a lot better than you would a complete stranger's. Having access to reviews of friends and family and people you choose to follow is why I built http://www.mybema.com http://www.mybema.com
- abrahamsen 13y agoThis is why Google did the right thing tying their reviews to their social network. If I know the person doing the review, I can put it in context. There are people whose opinion on wireless routers I'd listen to, and people I wouldn't. A lot of people got angry at the time, and publicly declared they wouldn't review any more. But that's not really a big loss. Their reviews might be the most qualified out there, but as long as they were anonymous I would be unable to put them in context, and they would be of very limited value to me.
- 001sky 13y agoThis just leads to survivorship bias. Its a shitty hack. A good review systems needs to encourage honesty, not discretion (selective revelation). The substitution of selective revelation for "honesty" is not a more honest or more reliable system. This is a common mistake, and one that marketers like to promote. As it works to their advantage: to manipulate the selection and revelation process, it helps to have a system in place such as the one you describe.
- abrahamsen 13y agoI don't understand a word you are saying.
- 001sky 13y agoAparently.
- TrainedMonkey 13y agoI think while that is one benefit of their move, it was not primary purpose. Google is data. They had different users on different services, now they are trying for unified account (Which is probably going to be G+ with everything else attached to it). In essence that is to allow people "convenient" access to all Google services with a single login. However, really it is to tie real people to their services (Easier to sell shit to real people) and to chain them to using Google. I know there are lots of other reasons, but I believe what I stated above to be main reasons that outline Google's long term strategic plan. Keep driving ad click through rates up by showing more and more custom ads to everyone.