7 ms·

With ZFS, he is better prepared than other WD and Sandisk SSD users. https://petapixel.com/2023/08/08/sandisk-portable-ssds-are-failing-so-frequently-we-can-no

by xet7 3y ago

With ZFS, he is better prepared than other WD and Sandisk SSD users. https://petapixel.com/2023/08/08/sandisk-portable-ssds-are-failing-so-frequently-we-can-no-longer-recommend-them/ https://petapixel.com/2023/08/08/sandisk-portable-ssds-are-f... https://www.theverge.com/22291828/sandisk-extreme-pro-portable-my-passport-failure-continued https://www.theverge.com/22291828/sandisk-extreme-pro-portab... https://news.ycombinator.com/item?id=37042587 https://news.ycombinator.com/item?id=37042587 https://www.theverge.com/23837513/western-digital-sandisk-ssd-corrupted-deleted-questions https://www.theverge.com/23837513/western-digital-sandisk-ss... https://news.ycombinator.com/item?id=37188736 https://news.ycombinator.com/item?id=37188736

anonuser123456 3y ago
I had two drives in my mirrored zpool die within 8 minutes of one another. Both HGST drives too. A very sad day. Thankfully I had been regularly zfs sending my contents to another site and lost very little data. ZFS is rad.
- anotherhue 3y ago
  > ZFS is rad. Typo: RAID
  - LargoLasskhyfv 3y ago
    I'd bet that was intentional, because https://www.urbandictionary.com/define.php?term=Rad https://www.urbandictionary.com/define.php?term=Rad
  - anonuser123456 3y ago
    You deserve all my upvotes :)
  - btown 3y ago
    Only 1/4 of data lost! Meaning still recoverable!
  - quickthrower2 3y ago
    Redundant Array of Disks (cost not specified)
    - askvictor 3y ago
      I always thought it was Independent Disks, though how one disk could be dependent on another is beyond me. Perhaps the I is redundant?
      - quickthrower2 3y ago
        Looking it up I think you are right. I thought it was inexpensive.
      - throw0101b 3y ago
        The original Patterson, Gibson, Katz paper has "Inexpensive": * https://www.cs.cmu.edu/~garth/RAIDpaper/Patterson88.pdf https://www.cs.cmu.edu/~garth/RAIDpaper/Patterson88.pdf * https://www.computerhistory.org/storageengine/u-c-berkeley-paper-catalyses-interest-in-raid/ https://www.computerhistory.org/storageengine/u-c-berkeley-p...
  - josephg 3y ago
    Repeat after me: RAID is not a backup. Sounds like it was the backup which saved the day here, not the raid array.
    - tjoff 3y ago
      He didn't have a raid array on his laptop, that would have saved him as well, with not even a second worth of data lost. And he could use his system without replacing any drive and he would have had a much quicker recovery and could have used the system during rebuild. Now, raid on a laptop is mostly reserved to bigger units. And authors setup is awesome as well, but single drive failure is one of the most common issues (especially where you make use of snapshots) you need your backup and that is exactly what raid solves. So try to do both :)
      - sofixa 3y ago
        > I had two drives in my mirrored zpool die within 8 minutes of one another. 2 drives in a mirrored zpool, this is the equivalent of RAID1.
        tjoff 3y ago
        Missed that context, meant regarding the article.
    - taskforcegemini 3y ago
      zfs send was used, so the failed zfs still had a hand in the positive outcome
- vardump 3y ago
  That's extremely unlikely. Could it have been the controller instead? Which HGST drives?
  - jms 3y ago
    Not extremely unlikely if they were identical drives from the same manufacturing batch. It's good practise to use diverse manufacturers or at least batches when adding disks to a raid array for just this reason.
  - repiret 3y ago
    I've frequently had drives in a RAID fail in rapid succession. If you buy a bunch of identical drives at the same time and put them in a RAID, then you can end up with: * They were manufactured in the same batch, maybe even one right after another on the same line. * As they were transported from manufacturer to OEM to you they were exposed the the same environmental conditions, right down to vibrations, humidity, and ambient EM environment. * As you use them, they continue to be exposed to the same environmental conditions, including power supply fluctuations and power inductively coupled into places it doesn't belong. * They see the same usage patterns. Depending on the RAID specifics, that might be right down to seeing the same disk locations seeing the same read and write volume. Its then not surprising if they fail at about the same time. The last machine I put together that I wanted to have high availability, I intentionally bought two different brand drives to put in the mirror to maximize the likelihood that they fail at very different times. Many years ago (c. 2003) the group I was working in inherited a massive 6U storage server with an insane number of 10k SCSI (it was before SAS was a thing) drives. We named it "hurricane" for the sound it made. After a few weeks of using it, the first drive failed. It rebuilt to a hot spare and we ordered and eventually installed a replacement. A few weeks later, another drive failed, and this time before it could finish rebuilding, two more drives in the RAID failed and its contents lost (but we had a good backup). We never used it again. For a while I used it as a coffee table, but then someone convinced me that was too tacky, and it got ewasted.
    - jjav 3y ago
      > If you buy a bunch of identical drives at the same time and put them in a RAID When setting up a new machine with zfs I intentionally buy drives from as many different brands and models as possible to spread the manufacturing defect risk.
- erhaetherth 3y ago
  Stop, you're scarring me! I have mirrored drives in a zpool. If a pair dies I lose 18 TB. My most important stuff is cloud-replicated but still...
  - densh 3y ago
    My pairs of mirrored drives come from 2 different manufacturers to prevent a common fault that happens at once to both drives.
  - orangepurple 3y ago
    You should probably be using RAID Z2 in that case which supports simultaneous 2 drive failure without a problem
    - benoliver999 3y ago
      I actually had this happen, and RAID Z2 saved me from a very long recovery process. I thought it might be the controller but a year on I've had no further issues. Sometimes drives do just go like lightbulbs.
  - teekert 3y ago
    3 2 1 backups, ddg it ;)
  - csydas 3y ago
    Ah, don't be scared. You're at least starting to think on your data and replicating your most important stuff elsewhere. I'd still recommend a non-cloud copy also, but you're probably okay :) Take it as a good impetus to catalog your data and find those extra replication options. Data protection does cost you a bit to do it "fully", but since I've worked on a backup solution before in a client facing way, trust me when I tell you that I've seen rather large businesses (a few you might even know as a household name) who have less consideration for their data than you've expressed in your 3 sentences :) So just figure out which of your data _truly_ needs to survive at all costs, get a solid setup with personally owned storage for the backups in combination with cloud storage, and you're probably fine.
- nicman23 3y ago
  i hope me mixing the models and buy times (i have been using the raidz expansion branch) will keep that raidz alive. it has become non economical / practical for me to backup everything
- alias_neo 3y ago
  I also had two mirrored drives fail simultaneously in my zpool a few days ago. There was nothing on them so I wasn't worried. WD Reds in my case; Using matched drives seems to be a very bad idea for mirrors. I'll probably replace them with two different brands. I also have a matched pair of HGST SAS Helium drives in the same backplane so hopefully I can catch those before they fail too if they're going to go at once, I _do_ have data on those.
  - miohtama 3y ago
    Is there a specific reason why the drives die at the same time? Electricity spike?
    - privong 3y ago
      If they're ~identical construction and being mirrored so that they have the same write/read pattern history, it could trigger the same failure mode simultaneously.
      - dizhn 3y ago
        More likely to be from the same bad batch too. There was a post with very detailed comments about this just a few days ago.
        alias_neo 3y ago
        Why bad? What's considered a good/bad lifetime for these? Mine had ~43000 power on hours, I don't know if that's good or bad for a WD Red (CMR) drive, but they weren't particularly heavily loaded, and their temps were good, so I'm fairly happy with how long they lasted (though longer would have been nice).
        dizhn 3y ago
        You're right it might be a natural end of life that coincides too.
j1elo 3y ago
Tangentially related, on topic with the Google search issues that The Verge complains about at https://www.theverge.com/22291828/sandisk-extreme-pro-portable-my-passport-failure-continued https://www.theverge.com/22291828/sandisk-extreme-pro-portab... > Google assumes you’re looking for product pages when you search for things like “4TB SanDisk SSD,” so news stories like ours and Ars Technica’s appear far down search results. Well, I've been test-driving the paid Kagi search engine, and this was an excellent opportunity to see if a different class of web search could produce different results... Sadly I'm afraid when all the web is flooded by praising articles, a different set of prioritization rules would still struggle to show different results: * First 5-6 results are from shops. * Then come some reviews, from: easeus [1], techpowerup [2], anandtech [3], consumerreviews [4]. None of them contain the word "fail". * Lastly, and this is the only actual improvement from google, there are some relevant search suggestions such as "sandisk 4tb ssd failure" and "sandisk 4tb ssd problems". Difficult to see (they are at the bottom of the page), but at least better than Google (where the word "fail" doesn't appear at all in the first results page). [1]: https://www.easeus.com/knowledge-center/sandisk-4tb-extreme-portable-ssd-v2-full-review.html https://www.easeus.com/knowledge-center/sandisk-4tb-extreme-... [2]: https://www.techpowerup.com/review/sandisk-ultra-3d-4-tb-ssd/ https://www.techpowerup.com/review/sandisk-ultra-3d-4-tb-ssd... [3]: https://www.anandtech.com/show/16892/sandisk-extreme-pro-crucial-x6-4tb-portable-ssds https://www.anandtech.com/show/16892/sandisk-extreme-pro-cru... [4]: https://consumerreviews.store/sandisk-4tb-extreme-portable-ssd-up-to-1050mb-s-usb-c-usb-3-2-gen-2-review/ https://consumerreviews.store/sandisk-4tb-extreme-portable-s...

by xet7 3y ago

anonuser123456 3y ago

I had two drives in my mirrored zpool die within 8 minutes of one another. Both HGST drives too. A very sad day. Thankfully I had been regularly zfs sending my contents to another site and lost very little data. ZFS is rad.

anotherhue 3y ago
> ZFS is rad. Typo: RAID
- LargoLasskhyfv 3y ago
  I'd bet that was intentional, because https://www.urbandictionary.com/define.php?term=Rad https://www.urbandictionary.com/define.php?term=Rad
- anonuser123456 3y ago
  You deserve all my upvotes :)
- btown 3y ago
  Only 1/4 of data lost! Meaning still recoverable!
- quickthrower2 3y ago
  Redundant Array of Disks (cost not specified)
  - askvictor 3y ago
    I always thought it was Independent Disks, though how one disk could be dependent on another is beyond me. Perhaps the I is redundant?
    - quickthrower2 3y ago
      Looking it up I think you are right. I thought it was inexpensive.
    - throw0101b 3y ago
      The original Patterson, Gibson, Katz paper has "Inexpensive": * https://www.cs.cmu.edu/~garth/RAIDpaper/Patterson88.pdf https://www.cs.cmu.edu/~garth/RAIDpaper/Patterson88.pdf * https://www.computerhistory.org/storageengine/u-c-berkeley-paper-catalyses-interest-in-raid/ https://www.computerhistory.org/storageengine/u-c-berkeley-p...
- josephg 3y ago
  Repeat after me: RAID is not a backup. Sounds like it was the backup which saved the day here, not the raid array.
  - tjoff 3y ago
    He didn't have a raid array on his laptop, that would have saved him as well, with not even a second worth of data lost. And he could use his system without replacing any drive and he would have had a much quicker recovery and could have used the system during rebuild. Now, raid on a laptop is mostly reserved to bigger units. And authors setup is awesome as well, but single drive failure is one of the most common issues (especially where you make use of snapshots) you need your backup and that is exactly what raid solves. So try to do both :)
    - sofixa 3y ago
      > I had two drives in my mirrored zpool die within 8 minutes of one another. 2 drives in a mirrored zpool, this is the equivalent of RAID1.
      - tjoff 3y ago
        Missed that context, meant regarding the article.
  - taskforcegemini 3y ago
    zfs send was used, so the failed zfs still had a hand in the positive outcome
vardump 3y ago
That's extremely unlikely. Could it have been the controller instead? Which HGST drives?
- jms 3y ago
  Not extremely unlikely if they were identical drives from the same manufacturing batch. It's good practise to use diverse manufacturers or at least batches when adding disks to a raid array for just this reason.
- repiret 3y ago
  I've frequently had drives in a RAID fail in rapid succession. If you buy a bunch of identical drives at the same time and put them in a RAID, then you can end up with: * They were manufactured in the same batch, maybe even one right after another on the same line. * As they were transported from manufacturer to OEM to you they were exposed the the same environmental conditions, right down to vibrations, humidity, and ambient EM environment. * As you use them, they continue to be exposed to the same environmental conditions, including power supply fluctuations and power inductively coupled into places it doesn't belong. * They see the same usage patterns. Depending on the RAID specifics, that might be right down to seeing the same disk locations seeing the same read and write volume. Its then not surprising if they fail at about the same time. The last machine I put together that I wanted to have high availability, I intentionally bought two different brand drives to put in the mirror to maximize the likelihood that they fail at very different times. Many years ago (c. 2003) the group I was working in inherited a massive 6U storage server with an insane number of 10k SCSI (it was before SAS was a thing) drives. We named it "hurricane" for the sound it made. After a few weeks of using it, the first drive failed. It rebuilt to a hot spare and we ordered and eventually installed a replacement. A few weeks later, another drive failed, and this time before it could finish rebuilding, two more drives in the RAID failed and its contents lost (but we had a good backup). We never used it again. For a while I used it as a coffee table, but then someone convinced me that was too tacky, and it got ewasted.
  - jjav 3y ago
    > If you buy a bunch of identical drives at the same time and put them in a RAID When setting up a new machine with zfs I intentionally buy drives from as many different brands and models as possible to spread the manufacturing defect risk.
erhaetherth 3y ago
Stop, you're scarring me! I have mirrored drives in a zpool. If a pair dies I lose 18 TB. My most important stuff is cloud-replicated but still...
- densh 3y ago
  My pairs of mirrored drives come from 2 different manufacturers to prevent a common fault that happens at once to both drives.
- orangepurple 3y ago
  You should probably be using RAID Z2 in that case which supports simultaneous 2 drive failure without a problem
  - benoliver999 3y ago
    I actually had this happen, and RAID Z2 saved me from a very long recovery process. I thought it might be the controller but a year on I've had no further issues. Sometimes drives do just go like lightbulbs.
- teekert 3y ago
  3 2 1 backups, ddg it ;)
- csydas 3y ago
  Ah, don't be scared. You're at least starting to think on your data and replicating your most important stuff elsewhere. I'd still recommend a non-cloud copy also, but you're probably okay :) Take it as a good impetus to catalog your data and find those extra replication options. Data protection does cost you a bit to do it "fully", but since I've worked on a backup solution before in a client facing way, trust me when I tell you that I've seen rather large businesses (a few you might even know as a household name) who have less consideration for their data than you've expressed in your 3 sentences :) So just figure out which of your data _truly_ needs to survive at all costs, get a solid setup with personally owned storage for the backups in combination with cloud storage, and you're probably fine.
nicman23 3y ago
i hope me mixing the models and buy times (i have been using the raidz expansion branch) will keep that raidz alive. it has become non economical / practical for me to backup everything
alias_neo 3y ago
I also had two mirrored drives fail simultaneously in my zpool a few days ago. There was nothing on them so I wasn't worried. WD Reds in my case; Using matched drives seems to be a very bad idea for mirrors. I'll probably replace them with two different brands. I also have a matched pair of HGST SAS Helium drives in the same backplane so hopefully I can catch those before they fail too if they're going to go at once, I _do_ have data on those.
- miohtama 3y ago
  Is there a specific reason why the drives die at the same time? Electricity spike?
  - privong 3y ago
    If they're ~identical construction and being mirrored so that they have the same write/read pattern history, it could trigger the same failure mode simultaneously.
    - dizhn 3y ago
      More likely to be from the same bad batch too. There was a post with very detailed comments about this just a few days ago.
      - alias_neo 3y ago
        Why bad? What's considered a good/bad lifetime for these? Mine had ~43000 power on hours, I don't know if that's good or bad for a WD Red (CMR) drive, but they weren't particularly heavily loaded, and their temps were good, so I'm fairly happy with how long they lasted (though longer would have been nice).
        dizhn 3y ago
        You're right it might be a natural end of life that coincides too.

j1elo 3y ago

Tangentially related, on topic with the Google search issues that The Verge complains about at https://www.theverge.com/22291828/sandisk-extreme-pro-portable-my-passport-failure-continued https://www.theverge.com/22291828/sandisk-extreme-pro-portab... > Google assumes you’re looking for product pages when you search for things like “4TB SanDisk SSD,” so news stories like ours and Ars Technica’s appear far down search results. Well, I've been test-driving the paid Kagi search engine, and this was an excellent opportunity to see if a different class of web search could produce different results... Sadly I'm afraid when all the web is flooded by praising articles, a different set of prioritization rules would still struggle to show different results: * First 5-6 results are from shops. * Then come some reviews, from: easeus [1], techpowerup [2], anandtech [3], consumerreviews [4]. None of them contain the word "fail". * Lastly, and this is the only actual improvement from google, there are some relevant search suggestions such as "sandisk 4tb ssd failure" and "sandisk 4tb ssd problems". Difficult to see (they are at the bottom of the page), but at least better than Google (where the word "fail" doesn't appear at all in the first results page). [1]: https://www.easeus.com/knowledge-center/sandisk-4tb-extreme-portable-ssd-v2-full-review.html https://www.easeus.com/knowledge-center/sandisk-4tb-extreme-... [2]: https://www.techpowerup.com/review/sandisk-ultra-3d-4-tb-ssd/ https://www.techpowerup.com/review/sandisk-ultra-3d-4-tb-ssd... [3]: https://www.anandtech.com/show/16892/sandisk-extreme-pro-crucial-x6-4tb-portable-ssds https://www.anandtech.com/show/16892/sandisk-extreme-pro-cru... [4]: https://consumerreviews.store/sandisk-4tb-extreme-portable-ssd-up-to-1050mb-s-usb-c-usb-3-2-gen-2-review/ https://consumerreviews.store/sandisk-4tb-extreme-portable-s...