12 ms·

In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field. It is especially unlikely to be the underlying cause of unexp

In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field. It is especially unlikely to be the underlying cause of unexpected and seemingly premature catastrophic SSD failure. Also, it is not possible for an SSD to decrease your usable storage space over time. That would break any partition table format or filesystem that expects an ordinary block device. It also wouldn't win you any significant increase unusable lifespan, because when your drive starts retiring large numbers of blocks, all of your blocks are on the verge of failure. (Though keep in mind that "failure" here is defined as "only able to retain data for one year" for consumer SSDs).

phkahler 5y ago
>> It also wouldn't win you any significant increase unusable lifespan, because when your drive starts retiring large numbers of blocks, all of your blocks are on the verge of failure. That's not true. Blocks degrade when written to. Most files are written once and read many times.
- slaymaker1907 5y ago
  You're forgetting that SSDs use virtual addressing so writes are distributed relatively equally over the whole drive. Even data that is written once and never changed logically will probably move around on the SSD due to garbage collection. The virtualization algorithms are kept relatively secret, but I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks.
  - wtallis 5y ago
    Yep, wear leveling actually works. On some SSDs, there are SMART indicators to tell you the average, max and min block erase counts, so you can see for yourself that the erase counts are all about the same across the entire drive.
  - phkahler 5y ago
    >> I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks. I hadn't considered that. Make sense. Better to move never-changing data to blocks that are still good but not worn out so those available writes can be utilized.
londons_explore 5y ago
> Also, it is not possible for an SSD to decrease your usable storage space over time. It totally could be possible. For Windows/Linux, you could have a daemon that creates a 'ssd_wear' file which grows in size as the SSD wears out, and messages to the SSD which physical blocks it occupies. Mac could obviously do a deeper integration directly into the OS. > all of your blocks are on the verge of failure A failed block still stores some information (in an information entropy sense). It's totally possible to store one blocks worth of information across two, four, or eight failed blocks (with extra error correction information). Combining these two techniques, an SSD should never fail. Instead it slows down and gets smaller. Sadly as far as I'm aware, no consumer drive vendor has implemented these.
- wtallis 5y ago
  You should read up on Zoned Storage for NVMe SSDs, and some of the more recently added error handling features like Get LBA Status. The gist is that it is only practical for the SSD to be in charge of determining what has failed-not the host, and that retrofitting NAND retirement into the traditional block device/LBA model is not worth the trouble when there are also other reasons to migrate to a different, more flash-friendly abstraction.
- pkaye 5y ago
  I used to work on SSD firmware. I did implement something like the above for a custom solution for caching. When the flash storage is running low on space, it reduced the reported storage capacity and notifies the host which will need to trim some data to bring it below the reported storage capacity. The striping of data across multiple blocks also has been implemented. You could fail an entire NAND die and it would still function though a little slower assuming you still have enough spare blocks.
azalemeth 5y ago
> It is especially unlikely to be the underlying cause of unexpected and seemingly premature catastrophic SSD failure. One of my graduate students experienced this, the hard way: writing up his thesis, his mum plugged in "the wrong USB C" charger. I don't know exactly what happened, but the board was utterly fried. The problem with the soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery. On the M1+ models, you literally can't do anything except semi-fix the motherboard as the flash and its controllers are integrated and the keys are cryptographically securely stored. This video [1] shows what data recovery is really like -- I don't often link to YT videos, but it is an hour long, involves an awful lot of BGA de-balling and re-balling and the scouring of spare chips from other donor dead machines, as Apple doesn't let manufacturers sell the chips separately and datasheets are all NDA'd. We're very much in the part of the curve where you'd need a lab and a lot of serious work to get data off these things without either sacrificial macs or luck (e.g. one blow voltage regulator). Of course, the flip side of this is that your data is really, really secure. You just have to be damn sure that you back it up correctly. [1] https://www.youtube.com/watch?v=110x5rMHCIw https://www.youtube.com/watch?v=110x5rMHCIw
- mschuster91 5y ago
  > You just have to be damn sure that you back it up correctly. Most somewhat serious NAS appliances offer being a TimeMachine server which is more than enough for most people.
- donkeyd 5y ago
  > soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery Well, that's kind of the point. > You just have to be damn sure that you back it up correctly. Exactly. And that's as easy as plugging in an external hard drive occasionally, or storing critical files in the cloud.
  - CelestialTeapot 5y ago
    >> soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery >Well, that's kind of the point. Not that I don't appreciate a security-minded platform, it just seems overkill for 99% of the people who'll purchase them, and do nothing but cause heartache when the internal SSD fails. And fail they do. it's rare, but I've had several SSDs (Toshiba and Crucial, if it matters) fail within 1-2 years of moderate usage. No warning there was an issue, the drives just disappeared one day and I was left looking for backups.
ooboe 5y ago
https://tesla-info.com/blog/tesla-mcu1-emmc-failure.php https://tesla-info.com/blog/tesla-mcu1-emmc-failure.php
- wtallis 5y ago
  Do you have a point? Linking to a story about an embedded system that experiences flash wearout is not really relevant here. You're trying to make a comparison against a far smaller and lower-quality storage module than typical consumer SSDs, subjected to a vastly different workload. There are no useful conclusions to draw about consumer PCs from that case.
omegalulw 5y ago
> In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field. +1. The part that fails on consumer SSDs is typically the controller.
zaphirplane 5y ago
> Though keep in mind that "failure" here is defined as "only able to retain data for one year" for consumer SSDs So the M1 Mac lasts a year before the owner has to shell for an apple authorised repair to replace the disk ? Me thinks that may not be the story for apple or dell What am I missing

phkahler 5y ago

>> It also wouldn't win you any significant increase unusable lifespan, because when your drive starts retiring large numbers of blocks, all of your blocks are on the verge of failure. That's not true. Blocks degrade when written to. Most files are written once and read many times.

slaymaker1907 5y ago
You're forgetting that SSDs use virtual addressing so writes are distributed relatively equally over the whole drive. Even data that is written once and never changed logically will probably move around on the SSD due to garbage collection. The virtualization algorithms are kept relatively secret, but I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks.
- wtallis 5y ago
  Yep, wear leveling actually works. On some SSDs, there are SMART indicators to tell you the average, max and min block erase counts, so you can see for yourself that the erase counts are all about the same across the entire drive.
- phkahler 5y ago
  >> I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks. I hadn't considered that. Make sense. Better to move never-changing data to blocks that are still good but not worn out so those available writes can be utilized.

londons_explore 5y ago

> Also, it is not possible for an SSD to decrease your usable storage space over time. It totally could be possible. For Windows/Linux, you could have a daemon that creates a 'ssd_wear' file which grows in size as the SSD wears out, and messages to the SSD which physical blocks it occupies. Mac could obviously do a deeper integration directly into the OS. > all of your blocks are on the verge of failure A failed block still stores some information (in an information entropy sense). It's totally possible to store one blocks worth of information across two, four, or eight failed blocks (with extra error correction information). Combining these two techniques, an SSD should never fail. Instead it slows down and gets smaller. Sadly as far as I'm aware, no consumer drive vendor has implemented these.

wtallis 5y ago
You should read up on Zoned Storage for NVMe SSDs, and some of the more recently added error handling features like Get LBA Status. The gist is that it is only practical for the SSD to be in charge of determining what has failed-not the host, and that retrofitting NAND retirement into the traditional block device/LBA model is not worth the trouble when there are also other reasons to migrate to a different, more flash-friendly abstraction.
pkaye 5y ago
I used to work on SSD firmware. I did implement something like the above for a custom solution for caching. When the flash storage is running low on space, it reduced the reported storage capacity and notifies the host which will need to trim some data to bring it below the reported storage capacity. The striping of data across multiple blocks also has been implemented. You could fail an entire NAND die and it would still function though a little slower assuming you still have enough spare blocks.

azalemeth 5y ago

> It is especially unlikely to be the underlying cause of unexpected and seemingly premature catastrophic SSD failure. One of my graduate students experienced this, the hard way: writing up his thesis, his mum plugged in "the wrong USB C" charger. I don't know exactly what happened, but the board was utterly fried. The problem with the soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery. On the M1+ models, you literally can't do anything except semi-fix the motherboard as the flash and its controllers are integrated and the keys are cryptographically securely stored. This video [1] shows what data recovery is really like -- I don't often link to YT videos, but it is an hour long, involves an awful lot of BGA de-balling and re-balling and the scouring of spare chips from other donor dead machines, as Apple doesn't let manufacturers sell the chips separately and datasheets are all NDA'd. We're very much in the part of the curve where you'd need a lab and a lot of serious work to get data off these things without either sacrificial macs or luck (e.g. one blow voltage regulator). Of course, the flip side of this is that your data is really, really secure. You just have to be damn sure that you back it up correctly. [1] https://www.youtube.com/watch?v=110x5rMHCIw https://www.youtube.com/watch?v=110x5rMHCIw

mschuster91 5y ago
> You just have to be damn sure that you back it up correctly. Most somewhat serious NAS appliances offer being a TimeMachine server which is more than enough for most people.
donkeyd 5y ago
> soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery Well, that's kind of the point. > You just have to be damn sure that you back it up correctly. Exactly. And that's as easy as plugging in an external hard drive occasionally, or storing critical files in the cloud.
- CelestialTeapot 5y ago
  >> soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery >Well, that's kind of the point. Not that I don't appreciate a security-minded platform, it just seems overkill for 99% of the people who'll purchase them, and do nothing but cause heartache when the internal SSD fails. And fail they do. it's rare, but I've had several SSDs (Toshiba and Crucial, if it matters) fail within 1-2 years of moderate usage. No warning there was an issue, the drives just disappeared one day and I was left looking for backups.

ooboe 5y ago

https://tesla-info.com/blog/tesla-mcu1-emmc-failure.php https://tesla-info.com/blog/tesla-mcu1-emmc-failure.php

wtallis 5y ago
Do you have a point? Linking to a story about an embedded system that experiences flash wearout is not really relevant here. You're trying to make a comparison against a far smaller and lower-quality storage module than typical consumer SSDs, subjected to a vastly different workload. There are no useful conclusions to draw about consumer PCs from that case.

omegalulw 5y ago

> In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field. +1. The part that fails on consumer SSDs is typically the controller.

zaphirplane 5y ago

> Though keep in mind that "failure" here is defined as "only able to retain data for one year" for consumer SSDs So the M1 Mac lasts a year before the owner has to shell for an apple authorised repair to replace the disk ? Me thinks that may not be the story for apple or dell What am I missing