16 ms·
Samsung Busted for Cheating TV Test Benchmarks
- haunter 4y agoNever stop recommending https://www.rtings.com/ https://www.rtings.com/ They cover more and more gadgets but they still do the best TV reviews
- bayindirh 4y agoI've recently chosen my first and only TV in the light of their reviews, and I'm happy and content for getting what I've read so far. Their "continuous reviews" are great. OTOH, TV market has evolved to something bizarre since I stopped watching TV. Now, every big player is rolling out an armada of TVs from low end to high end, and fanboys of these brands fight relentlessly about every itty bitty detail in forums, face to face, etc.. I know, details matter, but nothing is perfect and this bike-shedding is not healthy in my opinion. When I tell the brand and model of the TV I got, some of my friends counter "but brand X's model Y is actually better because..." with flaming eyes and with an instant enthusiasm. No, it's not. I've got the best TV for my needs and budget, and I'm happy. Thank you.
- philliphaydon 4y agoPeople seem to be very passionate about tv Brand’s. I like to look at TVs when I go to the store and personally just looking at them side by side for me visually I would tend to want to pick between LG or Panasonic. But my friend prefers Samsung but it always looks tooooo vibrant to me.
- Schroedingersat 4y agoI just want one without any smart features or preprocessing. Signal in via hdmi, rca, or radio waves. Attempt to reproduce exactly the image the signal describes out with no frames waiting in a buffer or other added latency.
- philliphaydon 4y agoI think a lot of us do. I just want a good image and hook up an Apple TV and a computer. I can’t stand using Android TV. I’ve never seen an example of it not being slow. And with ads now becoming a thing I don’t want it even more.
- bayindirh 4y agoI'm on the same page with you. Actually, I was planning to do the same thing, but the one I got turns on in 2 seconds flat, and there's no feel of lag. It works reasonably snappy, which doesn't make you say "C'mon!". I think it's quite nice. The video processor in that thing is very nice too. I'm using it with out of the box settings, and I don't feel the need to tweak anything, yet. The model I have doesn't have blatant ads (and it's not a Samsung), so I'm a happy camper.
- philliphaydon 4y agoI had a Panasonic using the old My Homescreen. That thing was fast. When I moved I sold the TV as the temp apartment had a TV. It was a Samsung TV and omg its is slower than any Android TV Box I've ever tried. Took ~40s before I could use the TV. In Taiwan my father in law has a Sony TV with Android, it's a tiny bit slow, but it's acceptable. There's no Ads on it yet. I'm yet to experience ad's but seems like alot of people are beginning to, makes me worried.
- ta988 4y agoYeah that's my go to for most hardware too. Got my screen from their recommendations and really happy with it.
- causality0 4y agoRtings is great for the data they provide, but they can have a but of a blind eye for other problems. For example, Rtings will tell you if a TV has bad contrast or poor audio, but they're not going to mention when the color is so uneven you can see a pattern on solid colors.
- waiwai933 4y agoI generally use Wirecutter for my reviews -- anyone able to, um, compare and review the two?
- onionisafruit 4y agoJust poke around rtings.com and you’ll see the difference. It reminds me a bit of what wirecutter was in it’s pre-nyt days before it became a click generator.
- waiwai933 4y agoI have done in the past, but I'm not sure what to make of the differences - let's take wireless keyboards as I'm currently thinking about getting one. If we look at Rtings[1], the first one it recommends is the Logitech MX Keys. On the Wirecutter[2], it first recommends the Logitech K380. To be clear, I've only skimmed both reviews right now. Both go on to recommend the other's first choice as a good alternative pick (the primary difference being whether compact or full-size is better for most people). So that tells me for at least one specific category, they're in agreement. Does Rtings go into a lot more detail about each keyboard? Absolutely. But that isn't necessarily useful to me--part of the reason I use the Wirecutter is so that I don't end up spending days trying to find the perfect item in a category[3]. I read the review, seeing if there are any caveats for its top pick that matter to me, and make a choice based on that. So what I don't want to do when buying something that both review is to read through both reviews, because either: (A) they agree, like on the wireless keyboard, in which case I've wasted a little bit of time, but in the grand scheme of things, it's fine, or (B) they disagree, because now I'm tempted to go through and weigh each detailed point which is something I'm trying to avoid! [1]: https://www.rtings.com/keyboard/reviews/best/by-type/wireless https://www.rtings.com/keyboard/reviews/best/by-type/wireles... [2]: https://www.nytimes.com/wirecutter/reviews/the-best-bluetooth-keyboard/ https://www.nytimes.com/wirecutter/reviews/the-best-bluetoot... [3]: https://xkcd.com/1036/ https://xkcd.com/1036/
- onionisafruit 4y agoI see where you are coming from. Sometimes I feel the same as you where I just want somebody to give me a recommendation and a few pros and cons. Other times I want to deep dive into the review. I think it boils down to how big of a commitment the purchase is.
- saxonww 4y agoIt's a good site, but be aware that they don't cover all the TV brands on the market. I don't see Philips or Panasonic on there, for example. The site is based in Quebec and I wonder if they are only reviewing sets that are available in Canada.
- JoshTriplett 4y agoPart of how they choose devices to cover (apart from popularity) is by request; if people asked them to cover those brands they likely would.
- Rebelgecko 4y agoInterestingly their "best LED TV" is a Samsung
- TingPing 4y agoBecause by most measures they make some of the best TVs. The complaints are just about software which nobody does indepth reviews of.
- jay_kyburz 4y agoYes but, now we know not to trust their ratings. They should review the product entirely, or they are wasting everybodies time.
- wolverine876 4y agoAlso Consumer Reports, who do highly scientific tests; have labs, engineers, etc.; and have decades of experience. Also, it's published by a non-profit established to serve consumers.
- akdor1154 4y agoHaving worked with Samsung's TV signage platform, i am amazed the duct tape for this hack stuck together long enough to be discovered in a working state. I know it's a big company, but some of the design and APIs were just an endless stream of WTFs.
- aluminussoma 4y agoI would only be surprised if the other manufacturers are NOT doing the same. We (collectively speaking) do the same at work. We optimize for what gets us promoted rather than what benefits the company. Companies optimize for revenue rather than a good user experience. This behavior of trying to game/cheat the benchmarks is entirely consistent with human behavior in other areas of life.
- jreese 4y agoOne of the best parts of being happy at a "terminal level" in FAANG is that I can easily pick what I work on for best impact on the teams around me (and the engineering org as a whole) without worrying about chasing a promotion. I know the levers I can pull to get a higher perf rating here and there, but I'm also free to focus on the "papercuts" knowing that I'm not sacrificing future promotions, because I just don't care about adding more responsibilities to my plate. I can do "the right thing" for my coworkers, and all I have to do is make a case for improving developer sentiment or efficiency to justify the work.
- aluminussoma 4y agoNo you can't. Source: I, too, was at a terminal level at a FAANG.
- oh_sigh 4y agoSure you can? You can even be a terminal L4 these days and just coast while getting paid like $250k/yr.
- jreese 4y agoI've been doing it for over six years now, so why not? Joined ten years ago yesterday and made it to IC5, and haven't had any desire for a promo since then, so I just work on we/hatever I think is the best thing I can spend my time on for the team, and it's been sufficient for "meets all" or better every half except one. Just pick the right team with the right manager who actually cares about you and what is actually both for you and your coworkers.
- gandalfian 4y agoYou sometimes bump into a professional TV reviewer on UK forums and suddenly realize when you think you are polling views from the top handful of sites, he works for most of them, so they are actually just his repeated opinion on one TV set...
- Ocha 4y agoNot surprised Samsung doing it. They consistently over promise and under deliver. I remember when buying their SSDs they promised support for HW encryption in future firmware update (Opal2/eDrive if I remember correctly) but they never actually delivered it. Since then I don’t trust anything Samsung promises or says.
- jiggawatts 4y agoThey still market this feature today, even though it is dead in the water. Hardware encryption was implemented so poorly that Microsoft said “enough is enough”, and simply dropped all such features from BitLocker. It is impossible to enable this feature on any supported version of Windows. (The straw that broke the camel’s back was apparently some vendors using “all zeroes” as the master encryption key.) Samsung should be charged with false advertising. If I wasn’t too lazy I would report them to the ACCC.
- Gigachad 4y agoHardware encryption should never have been trusted in the first place since it’s impossible to audit. The completely obvious thing happened where it’s incompetently implemented.
- KennyBlanken 4y agoThey've been caught cheating on phone benchmarks, too. They also hyped the hell out of Apple removing the headphone jack...and then did the same thing. Ditto for Apple removing the default charger; Samsung quietly followed suit not long after Apple. The whole headphone jack thing is a joke. Just buy the sub-$10 Apple lighting or USB-C headphone adapter. An adapter that tests better than more than half of the "high end" "audiophile" DACs. Mine now lives on the end of my one pair of wired headphones. It has been very rough, but somehow I have managed to persevere.
- thewebcount 4y ago> They also hyped the hell out of Apple removing the headphone jack...and then did the same thing. Ditto for Apple removing the default charger; Samsung quietly followed suit not long after Apple. I have a friend who worked at Apple then later ended up at Samsung. He said that any original idea at Samsung was immediately squashed and they were basically told when designing features to do whatever Apple did. So I’m not too surprised by that.
- verytrivial 4y agoThis sounds very similar to how various car manufacturers rigged their products to be benchmark aware. It did NOT turn out well for their shareholders.
- 1970-01-01 4y agoExcept this can't harm a single person, while Dieselgate's impact was estimated at dozens and dozens of people killed and hundreds injured.
- LegitShady 4y ago>Except this can't harm a single person of course it harmed people - people spent money on TV sets that do not perform as they expect because samsung was purposely cheating on the tests. All of their customers were potentially harmed, as well as Samsung's competitors.
- throwaway81523 4y agoThe whole concept of benchmarking a TV set sounds like something from a dystopian science fiction milieu.
- scrollaway 4y ago... huh? We've been benchmarking TVs for decades. What am I missing?
- prawn 4y agoWhy's that? Wouldn't there be practical things to measure like picture quality, colour consistency, power usage, speaker quality, etc?
- throwaway81523 4y agoThat stuff is measurement and is great, but "benchmarking" = measuring performance of features that don't belong in TV's.
- prawn 4y agoI'm not following. The article talks about benchmarking "display brightness, quality, power consumption", and the cheating relates to peak luminance. Aren't they pretty normal features for TVs?
- Dylan16807 4y agoResponse time is also very commonly benchmarked, and also belongs in TVs.
- ketralnis 4y agoDid you read the article? It doesn't talk about any of those things
- kcb 4y agoUnbiased reviews of consumer electronics is about the least dystopian thing I can think of.
- icod 4y agoI'm totally done with Samsung. Their TVs are overpriced and bad, their phones add undeletable bloatware, remove useful stuff like headphone/mic jack, have low battery life and capacity and they force you to buy the flagship model, if you don't you'll receive a really small battery that even in the flagship model is not large enough. If you don't buy Galaxy series you won't even be able to mirror output via USB. And software support is really short. Even if the hardware works, they want you to buy a new phone 5 years later. Likewise their TV OS is subpar and region locked, not that Android is any better or the other OSes. All the TVs right now are really bad and not worth the money. My price limit was 600€. I ordered 4 TVs and returned them all, the worst was a "TCL" TV which had such a weak CPU that it couldn't even decode a 2MB/s SD stream. Recently Samsung had a bad batch of SSDs and they sold them via Amazon and despite them knowing that the batch was bad it wasn't pulled, instead they lowered the price. One guy bought 8 , 5 died and 2 were in spare mode and despite them being from the same batch the refused to replace the 2. Imagine buying a 4TB SSD, copying your stuff over from the old HDD to have it die 2 months later.
- CoastalCoder 4y agoMany of us share your distaste for the issues your mentioned. The challenge is finding better alternatives. Do you have any recommendations?
- lern_too_spel 4y agoLG makea better TVs, appliances, phones, everything.
- effingwewt 4y agoNo, they do not. The TVs and the appliances are garbage. So garbage they still can't hire techs in LA. They pay you commission but only only on closed and profitable calls. They are almost never closed due to backorders of parts, especially boards, and especially now. Samsung and LG are both the same garbage in different packages. For TVs Id do your best to find a dumb one or use a monitor. For applicances it depends but stay away from Maytag and Whirlpool and Kitchenaid. WP bought out maytag and now both are garbage. Speed Queen makes amazing washers and dryers, kitchen applicances I'd try and go with Subzero for refrigerators and freezers, and ranges don't really matter just stay away from induction heating and dual or cabinet-microwave ranges. Just my $.02 after working on them and still being WP certified. IMO WP/Maytag and Amana are probably the worst. Samsung and LG not far behind, doubly so for their insane prices.
- jiggawatts 4y agoEvery time this comes up I feel obligated to add the following: 1) Samsung televisions spy on your mercilessly. They collect everything they can, including screenshots at regular intervals and send them back to Samsung so that they can sell this to "partners". They are not a FAANG and don't have anywhere near their level of security expertise. I have second-hand but reliable information about their security approach, and "laughable" doesn't begin to describe it. Assume that whatever you watch on a Samsung TV is being relayed to multiple state-level spy agencies. Ref: https://www.samsung.com/us/business/samsungads/resources/tv-ad-retargeting/ https://www.samsung.com/us/business/samsungads/resources/tv-... 2) Even if you buy a flagship Samsung television for five figures, it's virtually guaranteed that within a year or two they will release an update the slows it down to molasses. These things have controller boards with miniscule amounts of RAM in them, and they use the same firmware across all models. When a new model with more RAM is released, they add more features, causing all older models that self update to start swapping to the pagefile. I've seen several Samsung televisions across several model years and price points get slowed down until response times are up to 2 seconds for most actions. This is "unusable garbage" quality, for crazy money. NEVER buy a Samsung television, ever. You're treated as the product, not the customer, despite paying them a lot of money. PS: This isn't even the only display calibration cheating that they do! Practically all Samsung models use a "shop mode" that cranks up brightness to 11 and oversaturates the colours. Even in "normal" mode Samsung televisions have forced "enhancements" that can't be turned off. There is no "calibrated mode", because it would look like trash on these over-priced televisions.
- interpol_p 4y agoNever allow a TV or monitor to connect to your Wi-Fi network and you'll be able to avoid these two issues
- jiggawatts 4y agoWell yes, but then you're basically buying an expensive monitor and will need a second device to be the "smarts". I personally use an Apple TV for this, but in general this can be frustrating for users that just want their Smart TV to work without spying on them or implementing centrally triggered scheduled obsolescence. This is a bit like saying you should "never" turn on your new car's engine and should immediately swap it out for a third-party one, because every knows Manufacturer X leaves ticking time bombs in their engines. This is why consumer protection agencies are created by governments!
- hss 4y agoAre there any good quality dumb tv available in market anymore?
- deleted 4y ago[deleted]
- oynqr 4y agoGigabyte makes a "monitor" with the same panel as the LG OLEDs.
- whateveriforgot 4y agoIn Australia, I have been unable to find _any_ dumb TV's over 40" for sale any more. I've checked a variety of stores online and none offer them anymore. I managed to pick up a 50" 4K TV from 'Kogan' a few years back and was hoping to get a second (ideally larger) one. But alas. That's not an option. As alternatives. I believe it's possible to get sinage/display TV's which are still dumb (but much more expensive) and one could look at getting a huge 'computer monitor' as well.
- oxplot 4y agoThe problem is that none of these tests are regulatory tests. There is no approval process for monitor performance nor is there a penalty for not adhering to a standard (short of a class action suit). Heck, even when there was such a regulation, testing and penalty, VW saw fit to fake their diesel emission tests.
- ilrwbwrkhv 4y agoSony has been the only tv brand which has always been solid.
- chriscjcj 4y agoI wrote Samsung off a few years ago. I was at my dad's house and noticed that his Samsung TV would do this strange thing at the end of commercials for movies. At the end of most movie commercials, there is an all black screen with white text. When this screen would show up, the TV's back-light would suddenly dim and then brighten back up on the when it went away. I went online and found a forum of other folks noticing and complaining about the phenomenon. Apparently, this line of Samsung TVs had an algorithm that would watch for scenes that had a lot of black. Since the TV did a poor job of displaying blacks, it would try to dim the back-lighting so the blacks would look more black and less washed out. In the case of my dad's TV, the cure was worse than the disease. I searched the settings hoping to be able to disable the "feature" to no avail. I eventually found another forum offering up hacked firmware you could install that claimed to disable the feature. Not going to try that on my dad's TV, but I did decide that if these types of shenanigans are in Samsung's repertoire, I would be steering clear. OP article seem to confirm that they're still up to their chicanery which is sad.
- qu4z-2 4y agoAh, I see you have discovered the joys of "dynamic contrast". Just about every monitor I've bought recently has that "feature", but luckily it can usually be disabled. Unfortunately they still measure contrast with that feature enabled so it's like five-billion-to-one, allegedly.
- omoikane 4y agoThe article linked from the above has more details: https://www.flatpanelshd.com/news.php?subaction=showfull&id=1654235588 https://www.flatpanelshd.com/news.php?subaction=showfull&id=... It answered a question I was going to ask, which is "why don't they just run in benchmark mode all the time?" And the answer is apparently because the extra brightness can not be sustained without damaging the panel.
- aq3cn 4y agoI am glad that I didn't purchase their smart monitor called M7. It is sad that Samsung is fast losing their hard earned reputation. https://www.androidauthority.com/samsung-tv-benchmark-cheating-3177061/ https://www.androidauthority.com/samsung-tv-benchmark-cheati...