3 ms·
Part 6 in particular is a decent distillation of stuff to keep in mind that should apply across SSDs from multiple vendors. It would be nice to have benchmarks
by barrkel 12y ago
Part 6 in particular is a decent distillation of stuff to keep in mind that should apply across SSDs from multiple vendors. It would be nice to have benchmarks that validate the individual assertions, of course.
There's a lot of complexity going on behind the scenes of modern SSDs. Simplistic benchmarks as posted on most review sites often don't address real-world performance. The ones I pay most attention to are things like BootRacer; it's quite remarkable how often a drive will be slower at booting Windows than a competitor, even when it beats them in simple metrics like sequential / random reads / writes with different queue depths.
I can see the potential for firmware being optimized for modalities that crop up often in benchmarks but less often elsewhere: the aforementioned sequential / random reads and writes with different block sizes and queue depths. Doing well on benchmarks probably drives a bunch of sales. But detecting and switching modality may not be latency-free, and shortcuts taken to improve absolute performance in those modalities may harm global performance in a real-world IO mix.
- polskibus 12y agoDo you have any links that would show that an SSD was tuned solely for benchmark purposes? I know people have found such things in GPU market but I've never heard about such claims in SSD market.
- ugexe 12y agoI assume he states "potential" because he sees how it could be possible, not because he has proof of it.
- Kurtz79 12y ago"Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency." http://www.anandtech.com/show/9144/crucial-bx100-120gb-250gb-500gb-1tb-ssd-review/3 http://www.anandtech.com/show/9144/crucial-bx100-120gb-250gb...
- baruch 12y agoI've had that much acknowledged from SSD vendors, each one blames the others for the benchmark wars but they all know that benchmarks are out there and customers will chose the SSD based on the published benchmarks so they tune the firmware to make sure they don't look bad in the benchmarks. I was even told that an SSD was made to work consistently over the life span but drop some of these consistency work (i.e. slow downs at start of life) and only kick them in after so many power-on-hours so that it will not look bad in benchmarks. It's a fact of life that customers rely on benchmarks and that vendors cannot educate all customers so the masses out there need to see a good benchmark so that's what gets optimized for.
- pixpop 12y agoSorry I can't provide links. But I work for a large tech company, currently qualifying SSDs for a new project. We ran into serious bugs with drives from a particular well-known SSD manufacturer (certain workloads cause drives to become unresponsive until power is cycled) and were able to demand detailed root cause analysis. In at least two cases they uncovered new bugs in code that had been inserted purely to improve the numbers for 4k random writes on certain benchmarks. These are consumer drives, not enterprise, and are current production.
- deleted 12y ago[deleted]
- danielvinson 12y agoBootRacer tests are really inaccurate. The problem is that boot times are dependent on many things outside of the SSD, namely Chipset, drivers, and BIOS settings. Testing methodology for the manufacturer would be to disable all features in the BIOS for this test to try to reduce variables, which leads to unclear lower numbers.