6 ms·

We just lost 3TB of data on a SanDisk Extreme SSD

73 points by RadixDLT 3y ago

jqpabc123 3y ago
When it comes to reliability, simple solutions are usually the best. I still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates.
- valianteffort 3y ago
  Buy some blu-ray discs. Losing data is soul crushing and while blu-rays are not necessarily cheap they will outlive you and are easy to store/copy.
  - jqpabc123 3y ago
    Too slow --- takes too long to backup.
  - supertrope 3y ago
    Pressed discs sure. Consumer grade BD-R discs should only be considered data in transit.
    - BirAdam 3y ago
      There’s always MDISC, but you’d need a ton of them.
- ksec 3y ago
  >I still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates. You will still need to set up either BTRFS or ZFS to avoid silent Data corruption which is happening more often than most people know. And you have to set it up to scrub it from time to time. It isn't automatic. Arguably the cheapest solution that hit the above bar would be Synology DS223J at $180, their newest entry model with BTRFS support ( on a fairly old Linux Kernel, I think it is 5.10 that will never get updated again in the life time of the product ). You will still need to buy 2 HDD in redundant RAID and set it up yourself. And it is quite bulky as well. Hardly consumer friendly. And that is before 99.9999% on HN jumping on you, telling you need another backup for this NAS, or another off line, third location devices.... It is sad, not only just consumer. Judging by the number of comment here plenty in the tech industry simply uses an additional copy, or external SSD devices as backup. And I can understand why, but there simply isn't a simple, easy to use consumer solution that offer enough security for a reasonable price. And I have been stating this for more than a decade already. Your Kids's video are worth a lot more than you can imagine. Tried explaining it to your wife some of those video are gone.
  - joshcafe 3y ago
    My favorite trick for this lately has been to use a stripe of mirrored pairs, where each pair is a WD Gold and a WD Red Pro (of matching size). They're very similar drives (AFAICT, WD Gold is just a Red Pro but with a higher lifetime r/w rating) but it makes me feel a bit better that they're not from too similar of batches without having to actually temporally space out my purchases. I can buy 1 of each in the same cart. And then for truly unrecoverable data, I back it up to S3 Deep Archive which seems to be one of AWS's only reasonably priced offerings. Don't ask me how much it's cost in egress bandwidth to pull it all back though. I'm hoping if I end up needing it I'll be grateful enough for my foresight to justify paying it.
    - lowbloodsugar 3y ago
      So they can fail at the same time or did you make sure they are from different batches? Anyway, why use four disks from the same manufacturer???
      - em-bee 3y ago
        in my case i use 2.5" 5TB disks. there is only one model from one manufacturer making those. i bought one each from two different shops located in different parts of the country 5 years ago. i used them in a mirror raid. one of them died a year ago. then i got two more drives, one of which was second hand slightly used. the price was good and i figured it was a good way to ensure i'd get drives from two very different batches.
      - joshcafe 3y ago
        I'm not sure what exactly you mean - they're different lines of drives. And different SKUs on Amazon. The probability of me getting sibling drives out of that - and that's assuming the only difference between red pros and golds are binning - is infinitesimally small.
- somat 3y ago
  Heh, me too, but just a reminder raid is not a backup. raid can keep your data online in the face of some types of hardware failure. It won't protect against mistakes, malice, or disaster. backups will. What I do is build one good drive array then each machine gets a cheap local drive and can access the good drive via network and I don't have to spend too much putting redundancy in each machine. Now... "looks around embarrassedly" time to work on that backup system.
bad_alloc 3y ago
Uh... is one broken drive such a big issue? Hardware will fail, usually when it is very new or very old (Bathtub curve). Or are there more such events the article didn't mention?
- momirlan 3y ago
  yes there are, and they mention an Ars Technica story, as well as reddit users complaining. I lost 1TB on such a SSD I hardly used.
- RadixDLT 3y ago
  new drives should never crash or lose data, also WD and SanDisk lie through their teeth, there is no integrity.
  - worthless-trash 3y ago
    > new drives should never crash or lose data, In your small sample size maybe.
    - JonChesterfield 3y ago
      It'll be true for sufficiently short periods of time, or for ages if you never put any data on it. About 1/4 of hard drives I buy die in the first 24 hours since I unsportingly run badblocks on them to see whether they're on the start of the failure curve.
      - worthless-trash 3y ago
        Oh wow, that is -bad-. I'm usually being 10-12 and i think i've only had one or two over the last 5 years.
      - PrayagBhakar 3y ago
        What do you mean you run badblocks? Can you talk more about your drive testing process?
momirlan 3y ago
add me to the list of victims. "just" 1TB, but a drive that I hardly used a couple of times. horrible experience !
- BirAdam 3y ago
  I actually had 3x 2TB NVMe SSDs from them go bad… after just a few weeks.
lawgimenez 3y ago
Yes I bought a new WD 1TB a year ago and one day it was damaged beyond repair. I did not do anything with it. Shop technician's diagnosis was "several tiny scratches on the disk". Not sure what that meant, I assume the whole build of the drive is just bad? I just lost a 4 years worth of my kid's videos. I will never forgive WD. Edit: I just forgot to add that my old WD HD (bought around 2009-ish) is still working. But reading the contents will take hours. But the point is the quality of WD has gone downhill.
- gruez 3y ago
  >I just lost a 4 years worth of my kid's videos. I will never forgive WD. I'm sorry, but your fate was sealed the moment you decided that one copy was sufficient. Drives can fail randomly, so keeping valuable data on one without redundancy will virtually guarantee that you will lose data at some point in time. Just be thankful that you learned your lesson and it only cost 4 years worth of videos. If it happened 3 years down the line it could easily have been 7 years worth of videos.
  - worthless-trash 3y ago
    Drives exist in 3 states. About to die, dead and on fire. Plan accordingly.
  - gochi 3y ago
    Just stored 7 years worth of memories in the cloud, they can worry about redundancy!
    - lawgimenez 3y ago
      Ha! I am on iCloud Family max plan and I feel like it will get full 5 years from now.
      - JonChesterfield 3y ago
        Different failure mode. Try not to get locked out by Apple. Consider a local copy as well, hard drives are really cheap.
bombcar 3y ago
Many backups are mandatory - even if a drive never fails it can get lost or stolen.
- Wowfunhappy 3y ago
  Even with backups, drive failure rate matters! If you keep n backups, and a drive dies, then you will at least temporarily have n - 1 backups, which is presumably one less than you wanted.
Wowfunhappy 3y ago
Make sure to read the full article on this one, not just the beginning. I was initially rolling my eyes--yes, loosing data sucks, but all drives can die and there's such a thing as bad luck--but the issue really does appear to be larger than that.
j-bos 3y ago
I did not realize SanDisk used WD. Does anyone have an alternate storage brand to recommend?
- Shawnj2 3y ago
  Samsung
- rockemsockem 3y ago
  WD owns SanDisk, they bought them several years ago
londons_explore 3y ago
I blame tech journalism like the verge for this failure. SSD manufacturers are well aware of what their drives should do. But they are also aware that to get sales they need to win benchmarks, and that to do that they cut corners with data integrity. A benchmark of a drive should involve a heavy workload with a random drive hard and unplanned poweroff every 30 seconds. The drive should never lose a single fsync'ed byte. The drive should also be put in hot/cold cycling chambers. And in radiation chambers. And journalists should publish which drives meet their obligations despite all the adverse (yet still in spec) conditions they're put through. I bet no ssd for sale today could survive all 3 tests above for a month and still get okayish performance without data loss.
- londons_explore 3y ago
  Journalists could go even further. Have robustness tests where they drop the drive in salt water, shoot it with a gun, chuck it in a fire etc., And then see how much data can be recovered with data recovery tools.
- SemioticStandrd 3y ago
  …right. Don’t blame the manufacturer for making shitty drives, blame tech review sites for testing those drives with the tools available and reporting the results of those tests. This is some 4D level victim blaming shit. I’m almost impressed at the absurdity of this argument.
  - londons_explore 3y ago
    Tech review sites put extreme pressure on manufacturers. When the review sites only review one thing, corners get cut to make that one thing look good. Any honest manufacturer who did what they thought was best for the customer, not what got them the best reviews, went bankrupt long ago.
helf 3y ago
[dead]
qrohlf 3y ago
This exact SSD failed on me in a similar way. They also recently had a firmware issue that borked a bunch of drives, and they only released a fix tool for Windows users (I guess macOS/Linux users can pound sand?) I will say that the warranty service was quite good, but given the below-zero level of trust I have in SanDisk post WD acquisition, the replacement drive is going to be basically fulfilling the role of “paperweight and very oversized temporary USB drive for ephemeral sneakernet transfers”. Bought a pair of Samsung T7’s to replace it, no issues with them so far.
happytiger 3y ago
Many years ago we discovered that hard drives can be run in RAID configuration to avoid failure wiping out data. Bad drive runs happen. They’ve been happing a LOOONG time. If it’s important, dupe or raid. Otherwise, expect to lose the data. It’s just a matter of time. Lots of film and audio gear has a double recording option for just this reason. I’m sorry the author has to relearn this lesson the hard way!
mips_r4300i 3y ago
I have this exact drive; when plugged into my desktop PC it instantly wedges the USB root hub, causing USB to not work until I power cycle the entire machine. (Intel, W10) Yet the same drive works flawlessly in an Intel laptop. I think the m.2 to USB bridge chip inside might be the issue.

73 points by RadixDLT 3y ago

When it comes to reliability, simple solutions are usually the best. I still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates.

valianteffort 3y ago
Buy some blu-ray discs. Losing data is soul crushing and while blu-rays are not necessarily cheap they will outlive you and are easy to store/copy.
- jqpabc123 3y ago
  Too slow --- takes too long to backup.
- supertrope 3y ago
  Pressed discs sure. Consumer grade BD-R discs should only be considered data in transit.
  - BirAdam 3y ago
    There’s always MDISC, but you’d need a ton of them.
ksec 3y ago
>I still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates. You will still need to set up either BTRFS or ZFS to avoid silent Data corruption which is happening more often than most people know. And you have to set it up to scrub it from time to time. It isn't automatic. Arguably the cheapest solution that hit the above bar would be Synology DS223J at $180, their newest entry model with BTRFS support ( on a fairly old Linux Kernel, I think it is 5.10 that will never get updated again in the life time of the product ). You will still need to buy 2 HDD in redundant RAID and set it up yourself. And it is quite bulky as well. Hardly consumer friendly. And that is before 99.9999% on HN jumping on you, telling you need another backup for this NAS, or another off line, third location devices.... It is sad, not only just consumer. Judging by the number of comment here plenty in the tech industry simply uses an additional copy, or external SSD devices as backup. And I can understand why, but there simply isn't a simple, easy to use consumer solution that offer enough security for a reasonable price. And I have been stating this for more than a decade already. Your Kids's video are worth a lot more than you can imagine. Tried explaining it to your wife some of those video are gone.
- joshcafe 3y ago
  My favorite trick for this lately has been to use a stripe of mirrored pairs, where each pair is a WD Gold and a WD Red Pro (of matching size). They're very similar drives (AFAICT, WD Gold is just a Red Pro but with a higher lifetime r/w rating) but it makes me feel a bit better that they're not from too similar of batches without having to actually temporally space out my purchases. I can buy 1 of each in the same cart. And then for truly unrecoverable data, I back it up to S3 Deep Archive which seems to be one of AWS's only reasonably priced offerings. Don't ask me how much it's cost in egress bandwidth to pull it all back though. I'm hoping if I end up needing it I'll be grateful enough for my foresight to justify paying it.
  - lowbloodsugar 3y ago
    So they can fail at the same time or did you make sure they are from different batches? Anyway, why use four disks from the same manufacturer???
    - em-bee 3y ago
      in my case i use 2.5" 5TB disks. there is only one model from one manufacturer making those. i bought one each from two different shops located in different parts of the country 5 years ago. i used them in a mirror raid. one of them died a year ago. then i got two more drives, one of which was second hand slightly used. the price was good and i figured it was a good way to ensure i'd get drives from two very different batches.
    - joshcafe 3y ago
      I'm not sure what exactly you mean - they're different lines of drives. And different SKUs on Amazon. The probability of me getting sibling drives out of that - and that's assuming the only difference between red pros and golds are binning - is infinitesimally small.
somat 3y ago
Heh, me too, but just a reminder raid is not a backup. raid can keep your data online in the face of some types of hardware failure. It won't protect against mistakes, malice, or disaster. backups will. What I do is build one good drive array then each machine gets a cheap local drive and can access the good drive via network and I don't have to spend too much putting redundancy in each machine. Now... "looks around embarrassedly" time to work on that backup system.

bad_alloc 3y ago

Uh... is one broken drive such a big issue? Hardware will fail, usually when it is very new or very old (Bathtub curve). Or are there more such events the article didn't mention?

momirlan 3y ago
yes there are, and they mention an Ars Technica story, as well as reddit users complaining. I lost 1TB on such a SSD I hardly used.
RadixDLT 3y ago
new drives should never crash or lose data, also WD and SanDisk lie through their teeth, there is no integrity.
- worthless-trash 3y ago
  > new drives should never crash or lose data, In your small sample size maybe.
  - JonChesterfield 3y ago
    It'll be true for sufficiently short periods of time, or for ages if you never put any data on it. About 1/4 of hard drives I buy die in the first 24 hours since I unsportingly run badblocks on them to see whether they're on the start of the failure curve.
    - worthless-trash 3y ago
      Oh wow, that is -bad-. I'm usually being 10-12 and i think i've only had one or two over the last 5 years.
    - PrayagBhakar 3y ago
      What do you mean you run badblocks? Can you talk more about your drive testing process?

momirlan 3y ago

add me to the list of victims. "just" 1TB, but a drive that I hardly used a couple of times. horrible experience !

BirAdam 3y ago
I actually had 3x 2TB NVMe SSDs from them go bad… after just a few weeks.

lawgimenez 3y ago

Yes I bought a new WD 1TB a year ago and one day it was damaged beyond repair. I did not do anything with it. Shop technician's diagnosis was "several tiny scratches on the disk". Not sure what that meant, I assume the whole build of the drive is just bad? I just lost a 4 years worth of my kid's videos. I will never forgive WD. Edit: I just forgot to add that my old WD HD (bought around 2009-ish) is still working. But reading the contents will take hours. But the point is the quality of WD has gone downhill.

gruez 3y ago
>I just lost a 4 years worth of my kid's videos. I will never forgive WD. I'm sorry, but your fate was sealed the moment you decided that one copy was sufficient. Drives can fail randomly, so keeping valuable data on one without redundancy will virtually guarantee that you will lose data at some point in time. Just be thankful that you learned your lesson and it only cost 4 years worth of videos. If it happened 3 years down the line it could easily have been 7 years worth of videos.
- worthless-trash 3y ago
  Drives exist in 3 states. About to die, dead and on fire. Plan accordingly.
- gochi 3y ago
  Just stored 7 years worth of memories in the cloud, they can worry about redundancy!
  - lawgimenez 3y ago
    Ha! I am on iCloud Family max plan and I feel like it will get full 5 years from now.
    - JonChesterfield 3y ago
      Different failure mode. Try not to get locked out by Apple. Consider a local copy as well, hard drives are really cheap.

bombcar 3y ago

Many backups are mandatory - even if a drive never fails it can get lost or stolen.

Wowfunhappy 3y ago
Even with backups, drive failure rate matters! If you keep n backups, and a drive dies, then you will at least temporarily have n - 1 backups, which is presumably one less than you wanted.

Wowfunhappy 3y ago

Make sure to read the full article on this one, not just the beginning. I was initially rolling my eyes--yes, loosing data sucks, but all drives can die and there's such a thing as bad luck--but the issue really does appear to be larger than that.

j-bos 3y ago

I did not realize SanDisk used WD. Does anyone have an alternate storage brand to recommend?

Shawnj2 3y ago
Samsung
rockemsockem 3y ago
WD owns SanDisk, they bought them several years ago

londons_explore 3y ago

I blame tech journalism like the verge for this failure. SSD manufacturers are well aware of what their drives should do. But they are also aware that to get sales they need to win benchmarks, and that to do that they cut corners with data integrity. A benchmark of a drive should involve a heavy workload with a random drive hard and unplanned poweroff every 30 seconds. The drive should never lose a single fsync'ed byte. The drive should also be put in hot/cold cycling chambers. And in radiation chambers. And journalists should publish which drives meet their obligations despite all the adverse (yet still in spec) conditions they're put through. I bet no ssd for sale today could survive all 3 tests above for a month and still get okayish performance without data loss.

londons_explore 3y ago
Journalists could go even further. Have robustness tests where they drop the drive in salt water, shoot it with a gun, chuck it in a fire etc., And then see how much data can be recovered with data recovery tools.
SemioticStandrd 3y ago
…right. Don’t blame the manufacturer for making shitty drives, blame tech review sites for testing those drives with the tools available and reporting the results of those tests. This is some 4D level victim blaming shit. I’m almost impressed at the absurdity of this argument.
- londons_explore 3y ago
  Tech review sites put extreme pressure on manufacturers. When the review sites only review one thing, corners get cut to make that one thing look good. Any honest manufacturer who did what they thought was best for the customer, not what got them the best reviews, went bankrupt long ago.

helf 3y ago

[dead]

qrohlf 3y ago

This exact SSD failed on me in a similar way. They also recently had a firmware issue that borked a bunch of drives, and they only released a fix tool for Windows users (I guess macOS/Linux users can pound sand?) I will say that the warranty service was quite good, but given the below-zero level of trust I have in SanDisk post WD acquisition, the replacement drive is going to be basically fulfilling the role of “paperweight and very oversized temporary USB drive for ephemeral sneakernet transfers”. Bought a pair of Samsung T7’s to replace it, no issues with them so far.

happytiger 3y ago

Many years ago we discovered that hard drives can be run in RAID configuration to avoid failure wiping out data. Bad drive runs happen. They’ve been happing a LOOONG time. If it’s important, dupe or raid. Otherwise, expect to lose the data. It’s just a matter of time. Lots of film and audio gear has a double recording option for just this reason. I’m sorry the author has to relearn this lesson the hard way!

mips_r4300i 3y ago

I have this exact drive; when plugged into my desktop PC it instantly wedges the USB root hub, causing USB to not work until I power cycle the entire machine. (Intel, W10) Yet the same drive works flawlessly in an Intel laptop. I think the m.2 to USB bridge chip inside might be the issue.