4 ms·

In my experience, buying the “latest and greatest” hard drive is a recipe for disaster. They should put a sticker on the box that says “warning: will likely die

In my experience, buying the “latest and greatest” hard drive is a recipe for disaster. They should put a sticker on the box that says “warning: will likely die within one year.” Maybe I’ve just gotten unlucky over the years though. Can anyone else confirm/deny that the latest hard drives tend to be unreliable? (Cue sampling bias...) EDIT: The article is about an enterprise HD, but I meant to ask a general question about consumer-grade drives. Presumably enterprise drives are much more reliable, because reasons. (It would be fascinating to know those reasons though!)

mceachen 6y ago
If you look at the CrashPlan drive reports, that doesn't seem to be their experience (and I don't know of a current, larger, public hard drive reliability report). It seems that newer models trend with priors from that brand, with some outliers.
- sillysaurusx 6y ago
  Thank you for pointing out BackBlaze! https://www.backblaze.com/b2/hard-drive-test-data.html https://www.backblaze.com/b2/hard-drive-test-data.html https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1-2020/ https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1...
- mceachen 6y ago
  Oops. Pretend I said BackBlaze instead of CrashPlan.
hyperman1 6y ago
One reason: If you send a flush command to a consumer grade hd, it returns before the flush is done. This makes it look nicer on older HD benchmarking software. Enterprise hds return when the flush is complete, so databases acknowledge a commit only when it was actually succesfull.
- pkaye 6y ago
  > If you send a flush command to a consumer grade hd, it returns before the flush is done. So essentially its a no-op?
  - lukevp 6y ago
    Just speculating but perhaps the consumer ones return after the cache is populated and not after the data is durably on disk? That wouldn’t really be a no-op but would be much quicker for small writes at the expense of correctness in a power outage (although in the case of a DB the WAL entries are typically atomic so it would just be a rollback anyway).
  - dijit 6y ago
    Think of it like mongodb thinks of data: it's eventually consistent, maybe.
- polskibus 6y ago
  How can I find out which drives exactly have synchronous flush and which don't? Is it a widely publicized fact?
- Hello71 6y ago
  source? this sounds like an urban myth. there's no way that every single consumer grade hard drive "returns before the flush is done", given that: 1. consumer grade is not well-defined, 2. there are many different "flush commands", and it does not depend on whether a drive is "consumer grade", and 3. if this were the case, then filesystem and database corruption would be rampant, which it is eminently not. it would be impossible to write a remotely reliable high-performance database or filesystem if flushing were useless. professionals doing formal analyses like https://www.sqlite.org/psow.html https://www.sqlite.org/psow.html would be a complete waste of time, because everyone would be just praying anyways.
  - sandworm101 6y ago
    >> consumer grade is not well-defined WD has laid out a reasonable standard. Blue/Black/Green are consumer grades. Red is for NAS and Gold is for enterprise.
squarefoot 6y ago
> Maybe I’ve just gotten unlucky over the years though. It's not you, it was WD (and apparently other vendors too). https://blocksandfiles.com/2020/04/14/wd-red-nas-drives-shingled-magnetic-recording/ https://blocksandfiles.com/2020/04/14/wd-red-nas-drives-shin...
- pilsetnieks 6y ago
  Those aren't the "latest and greatest," and they didn't die per se, they're just slow for some workloads (some of those workloads that they're marketed for.)
- deleted 6y ago
  [deleted]
boring_twenties 6y ago
I had a few drives die in rapid succession, but then I realized I wasn't cooling the system properly. That was around 12 years ago, and I'm still using the drives I bought after that today. Maybe you have a similar problem?
bserge 6y ago
I stick with things that have been out for a few years. Consumers seem to be just unwilling beta testers of new hardware, later revisions are more reliable. Applies to everything from computers to cars to software heh
m463 6y ago
I've had friends who worked at hard drive manufacturers. They said companies can very accurately know the lifetime of a drive the day it is manufactured just from the vibration profile. enterprise > consumer drives. It also depends on the "customer" buying the drives - certain system manufacturers will get better drives. I think it's like restaurant owners getting better cuts of meat than you can get.

mceachen 6y ago

If you look at the CrashPlan drive reports, that doesn't seem to be their experience (and I don't know of a current, larger, public hard drive reliability report). It seems that newer models trend with priors from that brand, with some outliers.

sillysaurusx 6y ago
Thank you for pointing out BackBlaze! https://www.backblaze.com/b2/hard-drive-test-data.html https://www.backblaze.com/b2/hard-drive-test-data.html https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1-2020/ https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1...
mceachen 6y ago
Oops. Pretend I said BackBlaze instead of CrashPlan.

hyperman1 6y ago

One reason: If you send a flush command to a consumer grade hd, it returns before the flush is done. This makes it look nicer on older HD benchmarking software. Enterprise hds return when the flush is complete, so databases acknowledge a commit only when it was actually succesfull.

pkaye 6y ago
> If you send a flush command to a consumer grade hd, it returns before the flush is done. So essentially its a no-op?
- lukevp 6y ago
  Just speculating but perhaps the consumer ones return after the cache is populated and not after the data is durably on disk? That wouldn’t really be a no-op but would be much quicker for small writes at the expense of correctness in a power outage (although in the case of a DB the WAL entries are typically atomic so it would just be a rollback anyway).
- dijit 6y ago
  Think of it like mongodb thinks of data: it's eventually consistent, maybe.
polskibus 6y ago
How can I find out which drives exactly have synchronous flush and which don't? Is it a widely publicized fact?
Hello71 6y ago
source? this sounds like an urban myth. there's no way that every single consumer grade hard drive "returns before the flush is done", given that: 1. consumer grade is not well-defined, 2. there are many different "flush commands", and it does not depend on whether a drive is "consumer grade", and 3. if this were the case, then filesystem and database corruption would be rampant, which it is eminently not. it would be impossible to write a remotely reliable high-performance database or filesystem if flushing were useless. professionals doing formal analyses like https://www.sqlite.org/psow.html https://www.sqlite.org/psow.html would be a complete waste of time, because everyone would be just praying anyways.
- sandworm101 6y ago
  >> consumer grade is not well-defined WD has laid out a reasonable standard. Blue/Black/Green are consumer grades. Red is for NAS and Gold is for enterprise.

squarefoot 6y ago

> Maybe I’ve just gotten unlucky over the years though. It's not you, it was WD (and apparently other vendors too). https://blocksandfiles.com/2020/04/14/wd-red-nas-drives-shingled-magnetic-recording/ https://blocksandfiles.com/2020/04/14/wd-red-nas-drives-shin...

pilsetnieks 6y ago
Those aren't the "latest and greatest," and they didn't die per se, they're just slow for some workloads (some of those workloads that they're marketed for.)
deleted 6y ago
[deleted]

boring_twenties 6y ago

I had a few drives die in rapid succession, but then I realized I wasn't cooling the system properly. That was around 12 years ago, and I'm still using the drives I bought after that today. Maybe you have a similar problem?

bserge 6y ago

I stick with things that have been out for a few years. Consumers seem to be just unwilling beta testers of new hardware, later revisions are more reliable. Applies to everything from computers to cars to software heh

m463 6y ago

I've had friends who worked at hard drive manufacturers. They said companies can very accurately know the lifetime of a drive the day it is manufactured just from the vibration profile. enterprise > consumer drives. It also depends on the "customer" buying the drives - certain system manufacturers will get better drives. I think it's like restaurant owners getting better cuts of meat than you can get.