23 ms·

I’m looking into using sd cards for my backups. Easy to transport， easy to store elsewhere, not a challenging write-erase workload. Does anyone have experienc

I’m looking into using sd cards for my backups. Easy to transport， easy to store elsewhere, not a challenging write-erase workload. Does anyone have experience with this or have recent measurements? I do see many of the problems mentioned in the article below.

gritzko 2y ago
Funnily enough, I am doing these experiments in 2024. All seems to work in the short run. Please wait a year for the results :) P.S. USB flash drives failed miserably, except for maybe Samsung FIT. Very unreliable.
- dogsledsleddog 2y ago
  USB flash failed in terms of writes? I'm kind of curious about the story of longterm offline storage, vs occasional use since I think the controller needs to both have logic to actively refresh over months/years and occasionally be on to do so?
  - gritzko 2y ago
    Yes, I had a case of an SSD drive zeroing out over 2-3 years. Must also be true for SD cards.
st_goliath 2y ago
> I’m looking into using sd cards for my backups. probably not a good idea. Modern, high density SD cards will likely use not very reliable MLC NAND internally, pretty much requiring active maintenance by the software on the built-in controller. I once had an embedded board booting from MLC NAND that lost its entire u-boot environment after being without power for a year, just sitting on a shelf in my office. Similarly, at home, I have an SSD with Windows on it for occasionally playing games. It required extensive recovery after sitting in a desk drawer for a little over a year. I generally wouldn't really trust flash memory for offline backups.
- teddyh 2y ago
  Unpowered – i.e. unplugged – flash or solid state memory have been shown to lose its data after three months: <https://news.ycombinator.com/item?id=37144598 https://news.ycombinator.com/item?id=37144598>
  - st_goliath 2y ago
    Thanks for the extra references! I'll make sure to bookmark those and take a closer look later. Yeah, MLC already losing data after 3 months doesn't really surprise me that much. If you play with a raw MLC chip and turn off the ECC engine, the amount of bit flips you get from reading on some chips isn't even funny anymore. With one chip I tried, even with ECC, I managed to induce persistent bit flips fairly quickly just by reading the same physical page over-and-over. For extra fun: those bit flips can happen in another page that you didn't touch, because the bits are paired :-) What actually still does surprise me sometimes, is how reliable those things can work after all, if you just pile enough software abstraction and management on top.
    - dale_glass 2y ago
      You seem like the sort of person who'd know. Are there tools available to a normal person to take a look at this stuff? Eg, is there some way I can evaluate a not yet failing SD card? CDs had tools to check for C1 and C2 errors, is there a way to do the same with a SD card or SSD?
      - st_goliath 2y ago
        For SSD, I'd imagine you might get SMART data or something similar? I know that on Linux, there are mmc-utils, but I only used those with eMMC so far. I just tried to read out some general status information on uSD cards I have around: it didn't work for 2 Kingston and Acon cards (command timeout). But for a Swissbit card (industrial), it took some solid 10 seconds before reporting back that it's doing ok and is ready for data. Looks like it depends heavily on vendor support, possibly also undocumented/non-standard command sequences?
michaelt 2y ago
In my experience, Raspberry Pi DIYers are more than happy to talk about their experiences, but they've only got small sample sizes, they're buying things at retail, some will be relatively price-insensitive, and people who have problems might be particularly eager to complain. These people hate SD cards with a passion. The kind of people who are in a position to report on failure rates across a fleet of hundreds of SD cards (a) don't always have their employer's permission to talk about it, (b) don't want to publicly bitch about vendors they want to maintain a good relationship with, (c) can be very price sensitive depending on the product and (d) are often buying through special channels where vendors will do things like guarantee a specific controller firmware version. These people keep on releasing products that use SD cards.
Modified3019 2y ago
While not quite as tiny, an nvme enclosure with a real drive is likely going to be more reliable, assuming the USB port it’s connected to can actually supply the power it needs. The enclosures I’m currently fond of are from dockcase, which have a capacitor (PLP meaning power loss protection), so if power is cut, the drive is sent a “oh shit make things safe now” signal and has a few seconds to do so. I use the larger versions, but there’s a 2230 sized one https://www.dockcase.com/products/dockcase-pocket-m2-nvme-smart-2230-ssd-enclosure-grey https://www.dockcase.com/products/dockcase-pocket-m2-nvme-sm... That said, expect all drives to fail. Some you get a warning, others you don’t, so have multiple backups. I don’t recommend the cheap vendor branded Samsung SSDs on eBay, had too many of those die with the firmware version reporting as “ERRMOD” (error mode). Another good thing to note, is how to power cycle SSDs. Outside of Samsung drives, this has worked on a few drives over the years that decided to suddenly stop working: https://dfarq.homeip.net/fix-dead-ssd/ https://dfarq.homeip.net/fix-dead-ssd/

gritzko 2y ago

Funnily enough, I am doing these experiments in 2024. All seems to work in the short run. Please wait a year for the results :) P.S. USB flash drives failed miserably, except for maybe Samsung FIT. Very unreliable.

dogsledsleddog 2y ago
USB flash failed in terms of writes? I'm kind of curious about the story of longterm offline storage, vs occasional use since I think the controller needs to both have logic to actively refresh over months/years and occasionally be on to do so?
- gritzko 2y ago
  Yes, I had a case of an SSD drive zeroing out over 2-3 years. Must also be true for SD cards.

st_goliath 2y ago

> I’m looking into using sd cards for my backups. probably not a good idea. Modern, high density SD cards will likely use not very reliable MLC NAND internally, pretty much requiring active maintenance by the software on the built-in controller. I once had an embedded board booting from MLC NAND that lost its entire u-boot environment after being without power for a year, just sitting on a shelf in my office. Similarly, at home, I have an SSD with Windows on it for occasionally playing games. It required extensive recovery after sitting in a desk drawer for a little over a year. I generally wouldn't really trust flash memory for offline backups.

teddyh 2y ago
Unpowered – i.e. unplugged – flash or solid state memory have been shown to lose its data after three months: <https://news.ycombinator.com/item?id=37144598 https://news.ycombinator.com/item?id=37144598>
- st_goliath 2y ago
  Thanks for the extra references! I'll make sure to bookmark those and take a closer look later. Yeah, MLC already losing data after 3 months doesn't really surprise me that much. If you play with a raw MLC chip and turn off the ECC engine, the amount of bit flips you get from reading on some chips isn't even funny anymore. With one chip I tried, even with ECC, I managed to induce persistent bit flips fairly quickly just by reading the same physical page over-and-over. For extra fun: those bit flips can happen in another page that you didn't touch, because the bits are paired :-) What actually still does surprise me sometimes, is how reliable those things can work after all, if you just pile enough software abstraction and management on top.
  - dale_glass 2y ago
    You seem like the sort of person who'd know. Are there tools available to a normal person to take a look at this stuff? Eg, is there some way I can evaluate a not yet failing SD card? CDs had tools to check for C1 and C2 errors, is there a way to do the same with a SD card or SSD?
    - st_goliath 2y ago
      For SSD, I'd imagine you might get SMART data or something similar? I know that on Linux, there are mmc-utils, but I only used those with eMMC so far. I just tried to read out some general status information on uSD cards I have around: it didn't work for 2 Kingston and Acon cards (command timeout). But for a Swissbit card (industrial), it took some solid 10 seconds before reporting back that it's doing ok and is ready for data. Looks like it depends heavily on vendor support, possibly also undocumented/non-standard command sequences?

michaelt 2y ago

In my experience, Raspberry Pi DIYers are more than happy to talk about their experiences, but they've only got small sample sizes, they're buying things at retail, some will be relatively price-insensitive, and people who have problems might be particularly eager to complain. These people hate SD cards with a passion. The kind of people who are in a position to report on failure rates across a fleet of hundreds of SD cards (a) don't always have their employer's permission to talk about it, (b) don't want to publicly bitch about vendors they want to maintain a good relationship with, (c) can be very price sensitive depending on the product and (d) are often buying through special channels where vendors will do things like guarantee a specific controller firmware version. These people keep on releasing products that use SD cards.

Modified3019 2y ago

While not quite as tiny, an nvme enclosure with a real drive is likely going to be more reliable, assuming the USB port it’s connected to can actually supply the power it needs. The enclosures I’m currently fond of are from dockcase, which have a capacitor (PLP meaning power loss protection), so if power is cut, the drive is sent a “oh shit make things safe now” signal and has a few seconds to do so. I use the larger versions, but there’s a 2230 sized one https://www.dockcase.com/products/dockcase-pocket-m2-nvme-smart-2230-ssd-enclosure-grey https://www.dockcase.com/products/dockcase-pocket-m2-nvme-sm... That said, expect all drives to fail. Some you get a warning, others you don’t, so have multiple backups. I don’t recommend the cheap vendor branded Samsung SSDs on eBay, had too many of those die with the firmware version reporting as “ERRMOD” (error mode). Another good thing to note, is how to power cycle SSDs. Outside of Samsung drives, this has worked on a few drives over the years that decided to suddenly stop working: https://dfarq.homeip.net/fix-dead-ssd/ https://dfarq.homeip.net/fix-dead-ssd/