8 ms·

Wirecutter used to be good, but then it got bought by NYT. I'm surprised at how difficult it is to find legit reviewers in general. I don't trust these larger c

Wirecutter used to be good, but then it got bought by NYT. I'm surprised at how difficult it is to find legit reviewers in general. I don't trust these larger companies to give us non-biased reviews.

tomcam 3y ago
The big question is, how would good reviewers get paid? Affiliate relationships are a conflict of interest, but people won’t pay for reviews.
- sanderjd 3y ago
  Being a component of a popular media subscription bundle ... like the New York Times. That seems like the right model to me.
- throw9away6 3y ago
  Nobody really needs reviews of shitty products. A review site really only needs to write long form reviews of products they recommend due to quality or value. The affiliate stuff then really isn’t that much of a conflict if every product provides equal revenue.
  - Given_47 3y ago
    > every product provides equal revenue Though I’d imagine most places have differing affiliate structures depending on the brand
  - tomcam 3y ago
    Well yeah but… a lot of the time you don’t know up front that something is shitty, right?
- frognumber 3y ago
  A lot of people DO pay for reviews. Consumer Reports is an example. It's also possible to do transparent governance and have firewalls between reviewers and monatization. Few do, but it's a good idea.
  - tomcam 3y ago
    Consumer Reports appears to be a shell of its former self though. I like the firewall idea a lot but modern media are so lacking in credibility that I’m not convinced it would work.
- rodgerd 3y ago
  The Consumers' Institute of New Zealand has people sign up for membership, and that funds their reviewing and advocacy. I think that models like that, co-ops, and the like are the only real way to make it work. As soon as your reviews run on advertising, affiliate links, whatever, you've immediately got pressure to be mindful of revenue streams.
- patmcc 3y ago
  I think part of why I used to trust Wirecutter was all the links (that I saw, at the time) were just Amazon Affiliates - it mitigated the conflict of interest if I knew whichever of the 8 headphones I buy they're getting the same cut from Amazon. Why would they lie about which is best? But then with per-brand affiliate deals, yeah, I'm much more suspicious.
  - KingMob 3y ago
    > Why would they lie about which is best? If the affiliate cut was a percentage they have an incentive to recommend pricier options (assuming they can't predict the price-volume tradeoff of their readers). If the affiliate is a flat fee per item, they have an incentive to recommend cheaper options, so more people buy. And in either case, they have a disincentive not to recommend a category at all. None of these are complete trust-killers, I'm just pointing out that getting paid out by Amazon instead of the brands doesn't eliminate incentives. All it does is ensure any incentive is consistent.
    - daft_pink 3y ago
      I agree. I often wondered if they recommended pricier products before, because some things I bought were expensive. However, when they just were through amazon, they didn't have to have conversations with the company to get a commission. What happens if they test something and it's great, and the company making the great product refuses to give them a commission? If something is a good deal for them, will they not update the review, because the affiliate setup is making a lot of money and they don't want to adjust it? What happens if they don't like the affiliate system the company's website uses, because it doesn't track cookies well enough? Do they have to call the company after the test and have them setup a specific affiliate system? It just seems like a terrible idea and there all these conflicts when they are negotiating a side deal with the company they review vs just collecting a consistent cut from a large marketplace like amazon.
krackers 3y ago
Consumerlabs, consumer reports, rtings, and which.co.uk seem to be the only standardized independent review bodies I'm aware of.
- bombcar 3y ago
  Consumer reports at least is known to be gamed (and they have specific review methodology you need to understand and interpret). It’s hard
  - rootusrootus 3y ago
    > Consumer reports at least is known to be gamed I'd be interested to know more about this, but I haven't found much discussion of it when I search. My google fu may be weak today.
    - gandalfgreybeer 3y ago
      Not related to the topic on hand but I’ve felt my google fu diminish significantly over the past years. Not sure if it’s me or my google intuition just not matching the current algorithms
      - Baeocystin 3y ago
        It's not you, it's google. Seriously.
      - seventytwo 3y ago
        It’s google. Try Kagi.
    - somat 3y ago
      On the whole I think consumer reports is one of the better review organizations out there. but for a hint of reader betrayal read up on what they did to the suzuki samurai. It is not exactly cut and dried, but it was found that consumer reports was testing to get a specific result. And due to the trust that people put in consumer reports that result killed the market for the samurai in the US.
thrashh 3y ago
I used to think they were good but I think they were only good on paper, even before they were bought by NYT. I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot. I feel like Wirecutter just bought a bunch of stuff and compared things superficially.
- LorenPechtel 3y ago
  It's very hard to test durability.
- Baeocystin 3y ago
  I only read Wirecutter when Ars Technica would host their occasional article, and came to the same conclusion as you. Any time they covered something that I had experience with, it was painfully obvious that they had barely used the damn thing. If you were in to their house style, they picked things that fit in that sense, but a solid meh for the review side of things.
- joshstrange 3y ago
  > I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot. The other thing that happened more than few times is once the product got reviewed the producer would cut corners and start shipping a vastly inferior product since they knew they could coast on the WC recommendation and increase their profit margin even further. I'm sure some of the corner-cutting was due to increased sales (not necessarily malicious) but a lot of it seemed to come 100% from greed.
firecall 3y ago
They don't seem to get surfaced in Google anymore either. At least, not for me over here in Australia! Haven't looked at Wirecutter in the longest time!
lhl 3y ago
I dunno, I gave up on Wirecutter for anything I actually cared about almost 10 years ago - I found that for anything that I had actually done even basic research on, the Wirecutter was always wrong (and not just like sort of wrong, but like completely missing the point/out to lunch wrong): https://randomfoo.net/2014/03/22/the-wirecutter-is-always-wrong https://randomfoo.net/2014/03/22/the-wirecutter-is-always-wr...

Wirecutter used to be good, but then it got bought by NYT. I'm surprised at how difficult it is to find legit reviewers in general. I don't trust these larger companies to give us non-biased reviews.

tomcam 3y ago

The big question is, how would good reviewers get paid? Affiliate relationships are a conflict of interest, but people won’t pay for reviews.

sanderjd 3y ago
Being a component of a popular media subscription bundle ... like the New York Times. That seems like the right model to me.
throw9away6 3y ago
Nobody really needs reviews of shitty products. A review site really only needs to write long form reviews of products they recommend due to quality or value. The affiliate stuff then really isn’t that much of a conflict if every product provides equal revenue.
- Given_47 3y ago
  > every product provides equal revenue Though I’d imagine most places have differing affiliate structures depending on the brand
- tomcam 3y ago
  Well yeah but… a lot of the time you don’t know up front that something is shitty, right?
frognumber 3y ago
A lot of people DO pay for reviews. Consumer Reports is an example. It's also possible to do transparent governance and have firewalls between reviewers and monatization. Few do, but it's a good idea.
- tomcam 3y ago
  Consumer Reports appears to be a shell of its former self though. I like the firewall idea a lot but modern media are so lacking in credibility that I’m not convinced it would work.
rodgerd 3y ago
The Consumers' Institute of New Zealand has people sign up for membership, and that funds their reviewing and advocacy. I think that models like that, co-ops, and the like are the only real way to make it work. As soon as your reviews run on advertising, affiliate links, whatever, you've immediately got pressure to be mindful of revenue streams.
patmcc 3y ago
I think part of why I used to trust Wirecutter was all the links (that I saw, at the time) were just Amazon Affiliates - it mitigated the conflict of interest if I knew whichever of the 8 headphones I buy they're getting the same cut from Amazon. Why would they lie about which is best? But then with per-brand affiliate deals, yeah, I'm much more suspicious.
- KingMob 3y ago
  > Why would they lie about which is best? If the affiliate cut was a percentage they have an incentive to recommend pricier options (assuming they can't predict the price-volume tradeoff of their readers). If the affiliate is a flat fee per item, they have an incentive to recommend cheaper options, so more people buy. And in either case, they have a disincentive not to recommend a category at all. None of these are complete trust-killers, I'm just pointing out that getting paid out by Amazon instead of the brands doesn't eliminate incentives. All it does is ensure any incentive is consistent.
  - daft_pink 3y ago
    I agree. I often wondered if they recommended pricier products before, because some things I bought were expensive. However, when they just were through amazon, they didn't have to have conversations with the company to get a commission. What happens if they test something and it's great, and the company making the great product refuses to give them a commission? If something is a good deal for them, will they not update the review, because the affiliate setup is making a lot of money and they don't want to adjust it? What happens if they don't like the affiliate system the company's website uses, because it doesn't track cookies well enough? Do they have to call the company after the test and have them setup a specific affiliate system? It just seems like a terrible idea and there all these conflicts when they are negotiating a side deal with the company they review vs just collecting a consistent cut from a large marketplace like amazon.

krackers 3y ago

Consumerlabs, consumer reports, rtings, and which.co.uk seem to be the only standardized independent review bodies I'm aware of.

bombcar 3y ago
Consumer reports at least is known to be gamed (and they have specific review methodology you need to understand and interpret). It’s hard
- rootusrootus 3y ago
  > Consumer reports at least is known to be gamed I'd be interested to know more about this, but I haven't found much discussion of it when I search. My google fu may be weak today.
  - gandalfgreybeer 3y ago
    Not related to the topic on hand but I’ve felt my google fu diminish significantly over the past years. Not sure if it’s me or my google intuition just not matching the current algorithms
    - Baeocystin 3y ago
      It's not you, it's google. Seriously.
    - seventytwo 3y ago
      It’s google. Try Kagi.
  - somat 3y ago
    On the whole I think consumer reports is one of the better review organizations out there. but for a hint of reader betrayal read up on what they did to the suzuki samurai. It is not exactly cut and dried, but it was found that consumer reports was testing to get a specific result. And due to the trust that people put in consumer reports that result killed the market for the samurai in the US.

thrashh 3y ago

I used to think they were good but I think they were only good on paper, even before they were bought by NYT. I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot. I feel like Wirecutter just bought a bunch of stuff and compared things superficially.

LorenPechtel 3y ago
It's very hard to test durability.
Baeocystin 3y ago
I only read Wirecutter when Ars Technica would host their occasional article, and came to the same conclusion as you. Any time they covered something that I had experience with, it was painfully obvious that they had barely used the damn thing. If you were in to their house style, they picked things that fit in that sense, but a solid meh for the review side of things.
joshstrange 3y ago
> I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot. The other thing that happened more than few times is once the product got reviewed the producer would cut corners and start shipping a vastly inferior product since they knew they could coast on the WC recommendation and increase their profit margin even further. I'm sure some of the corner-cutting was due to increased sales (not necessarily malicious) but a lot of it seemed to come 100% from greed.

firecall 3y ago

They don't seem to get surfaced in Google anymore either. At least, not for me over here in Australia! Haven't looked at Wirecutter in the longest time!

lhl 3y ago

I dunno, I gave up on Wirecutter for anything I actually cared about almost 10 years ago - I found that for anything that I had actually done even basic research on, the Wirecutter was always wrong (and not just like sort of wrong, but like completely missing the point/out to lunch wrong): https://randomfoo.net/2014/03/22/the-wirecutter-is-always-wrong https://randomfoo.net/2014/03/22/the-wirecutter-is-always-wr...