5 ms·

Has anyone got any info on the long term reliability of SD cards? Could they be used for archival for 10+ years? What about 100?

tw04 10y ago
Flash storage in general is very poor for long-term storage. Current leakage occurs and if they're just sitting on a shelf there's no way for them to self-correct. Your best bet is still tape or optical (think bluray) media.
- ekianjo 10y ago
  any recommendation for tape recording devices these days, that will be still supported 10 years from now ?
  - taeric 10y ago
    I would think you could get basic support for any device from Linux. Question is really, why not offshore the device support to someone else entirely? Hosted backups are likely at least as reliable as anything you can do. Right?
  - apaprocki 10y ago
    I restored my 22 year old BBS off Colorado QIC tapes by jury rigging an old mini-ATX board with a floppy tape drive and installing Win98 on it. They read perfectly. Of course you can always pay someone to dump the data for you, but where's the fun in that?! I think any off the shelf modern tape will be just fine in 10 years.
    - agumonkey 10y ago
      Ha, just tried to read a Colorado tape backup from 98 (tape might be from 96 even, HP T1000 drive) and it ran like a charm. I didn't even realize it was an 18 yo storage until reading your comment. Even at floppy speed, copying the 400MB wasn't that long, and it's lovely to hear the cute drive noise (for retro lovers).
  - ghshephard 10y ago
    There is some evidence to suggest that these DVDs have good archival support, and I can guarantee you that DVD readers will still be abundant 10 years from now: http://www.mdisc.com/ http://www.mdisc.com/
    - codezero 10y ago
      I had about 1000 DVDs in storage of backed up satellite data. At one point I had to restore the backup using the DVDs and it was about ~8 years into the mission so I was worried that the data would be no good on the earliest discs. It turned out to be totally fine, as far as I could tell. None of the early data had SHA/MD5 sums (I added this later), so there's no guarantee the earlier discs weren't somewhat corrupted, but they were definitely readable. Back then, I had assumed they wouldn't have that long of a life, but as you seem to have found, and I discovered, once written, optical media seems to have a very long shelf life.
      - ghshephard 10y ago
        Most Optical Media degrades almost immediately, depending on storage environment, lasting as little as two two years before they start to degrade/aren't readable (depending on the dye and substrate). The MDISC is archival format - designed to last a minimum of 100 years when stored under reasonable conditions, and up to 1000 years if stored in archival (cool, dark) environments. They are essential carving physical holes into a stone like substrate. The verification tests are really quite impressive: http://www.esystor.com/images/China_Lake_Full_Report.pdf http://www.esystor.com/images/China_Lake_Full_Report.pdf
  - protomyth 10y ago
    An LTO drive is good long term since it's guaranteed to write 1 generation back and read 2 generations back. Very easy to use once you get the routine down. As always, remember to do a restore to prove your backup is an actual backup.
    - ekianjo 10y ago
      Thanks. I checked the prices and a LTO drive is... expensive. I would have thought a tape recording device would be cheap by now.
      - protomyth 10y ago
        Nope, but they do last a long time compared to other drives. We have stuff that got retired after 10 years (still working, we just upgraded standards). LTO-7 is out, so you might start picking up deals on older models.
  - ars 10y ago
    Store both the tape and the device to read it. PCI/IDE/SATA interfaces are so ubiquitous that even decades from now I fully expect you to be able to use them. If nothing else, via some kind of adapter.
    - varjag 10y ago
      Decades is optimistic for a tape reader or any electronics really. Caps dry out, rubber parts degrade and RoHS solder doesn't help the life expectancy. It might still work, it might not.
  - tw04 10y ago
    LTO is the industry standard. You'll have no issues finding an LTO tape drive 10 years from now on ebay, and if for some reason you did, Kroll will still be able to recover it.
- codezero 10y ago
  I'd like to see some solid data on this. I used to think that DVD+/-R and DVD+/-RW had a shelf life of <10 years (this was ~10 years ago) and I can't recall anyone asserting they lasted longer. After running a long term program where backups were written to DVD+RW discs, and seeing them last > 10 years, I was impressed. Now I'm finding sources that say that the post-write self-life of optical media is on the order of 10s to 100s of years. Anyhow, it just makes me wonder what the practical shelf-life is, and what the underlying media "does" over time :)
  - tw04 10y ago
    It breaks down like any material. The easiest way to sample this would be the REALLY cheap cd-r discs you could get in the late 90s. If you left one of those sitting on your dash, they would borderline melt in the sun... and for sure start peeling off the plastic if they sat on your dash for a couple hours. Same thing with newer media - if you keep it in a climate controlled environment the sky is the limit. If you abuse it, it's not going to see anywhere near the quoted life.
  - digi_owl 10y ago
    Iirc, RWs las longer than plain Rs. Something to do with there being a second layers between the stprage surface and the environment.
  - Amezarak 10y ago
    On the other hand, I've never gotten writable optical media to last longer than three years, despite storing them in a cool dark closet. Did you compare checksums on the data you wrote?
    - codezero 10y ago
      On the earliest data, nope! Very likely I had some or many flipped bits. I did have the backups on multiple discs and comparing the two they had the same data so I guess that's almost as good.
tangue 10y ago
Anecdotal evidence : As an amateur photographer SD cards are the least reliable storage. ( they are flash memory after all...)
- codezero 10y ago
  How did this manifest? SD cards that just weren't recognized over time? Do you have any other details? Did you stick to expensive brand name SD cards? Did you write to them often, or read from them often? Super curious what the usage pattern was.
  - tangue 10y ago
    Yes they're not recognised after a while. To avoid mechanical failures I'm unloading my files straight from my camera via USB and erase the card from the menu. Here's some things I've noticed - They failed more often when I'm shooting video in full HD or 4k (less than a year). When I'm shooting films I can erase a card 6-8 times in a weekend. - For Photos I have cards that last for two-three years, shooting in Raw and erasing the card approx. twice a month. - At first I tried Samsung Pro but it fails equally. I'm using Transcend Extreme Pro but in terms of reliability they're the same things than the budget Transcend imho (the speed is better that's the only reason why I buy them). I had bad experiences with Sandisk cards
Eire_Banshee 10y ago
Depends on how often you write to it
wmf 10y ago
SSDs are only rated for one year of retention with power off; I would expect SD cards to be worse. http://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention http://www.anandtech.com/show/9248/the-truth-about-ssd-data-... In theory flash can be use for archival but it would have to be refreshed periodically.
- digi_owl 10y ago
  Yeiks. I wonder how many will be burned by that...
- SmellyGeekBoy 10y ago
  FWIW, I found 4 old SD cards (16MB - 256MB, various manufacturers) in my attic last week that definitely haven't been used in 5 or more years, and the photos on them all copied over to my PC just fine. I'm guessing the much lower density of the smaller capacity cards helps a lot in this regard.
userbinator 10y ago
1TB (8Tbit) sounds almost certainly like it's going to be at least 2-bit if not 3-bit MLC, maybe even 4-bit. The endurance and retention numbers for 3-bit MLC are already pretty dismal in comparison to even 2-bit MLC, and the only way they've been able to convince consumers to buy this stuff is by increasing the capacity and multiplying by it. The cells are still intrinsically more fragile, and increasing the bits-per-cell increases capacity only multiplicatively while retention and endurance decrease exponentially; theoretically, to a very rough approximation, 4 bits per cell will have 4 times the capacity but only 1/16th the endurance and retention of 1 bit per cell. In practice, it's somewhat worse. Instead of a 1TB card using 4bpc flash that may offer 1K cycles per cell (1PB total writes), I'd rather have a 256GB SLC (1bpc, "old-school" flash) card with 16K endurance (4PB total writes). Both take up the same area (thus theoretically cost) and the SLC is in fact simpler in many ways to use because it doesn't really need such elaborate "management", but unfortunately in practice SLC is priced astronomically higher than the technology would suggest. I'm not much for conspiracy theories, but it sometimes makes me really think if the industry is deliberately misleading us into the worse choice in terms of reliability, just so they can sell more. This is one of my big gripes about the flash memory industry --- there is a lot of marketing, a lot of fluff. But if you look at the facts it's clear that no one considering the tradeoffs logically would've ever thought MLC to be in any way better. It can certainly be two, three, or four times the density of SLC, but it doesn't even last a half, third, or fourth as long.
- wmf 10y ago
  MLC lasts long enough and it is significantly cheaper. That's very logical.
  - userbinator 10y ago
    The problem is "long enough" is not... and the manufacturers are continually trying to make that shorter and try to convince you it's "long enough". IMHO a few years is NOT "long enough" for a nonvolatile storage device. A few decades is more reasonable.
- keshet 10y ago

tw04 10y ago

Flash storage in general is very poor for long-term storage. Current leakage occurs and if they're just sitting on a shelf there's no way for them to self-correct. Your best bet is still tape or optical (think bluray) media.

ekianjo 10y ago
any recommendation for tape recording devices these days, that will be still supported 10 years from now ?
- taeric 10y ago
  I would think you could get basic support for any device from Linux. Question is really, why not offshore the device support to someone else entirely? Hosted backups are likely at least as reliable as anything you can do. Right?
- apaprocki 10y ago
  I restored my 22 year old BBS off Colorado QIC tapes by jury rigging an old mini-ATX board with a floppy tape drive and installing Win98 on it. They read perfectly. Of course you can always pay someone to dump the data for you, but where's the fun in that?! I think any off the shelf modern tape will be just fine in 10 years.
  - agumonkey 10y ago
    Ha, just tried to read a Colorado tape backup from 98 (tape might be from 96 even, HP T1000 drive) and it ran like a charm. I didn't even realize it was an 18 yo storage until reading your comment. Even at floppy speed, copying the 400MB wasn't that long, and it's lovely to hear the cute drive noise (for retro lovers).
- ghshephard 10y ago
  There is some evidence to suggest that these DVDs have good archival support, and I can guarantee you that DVD readers will still be abundant 10 years from now: http://www.mdisc.com/ http://www.mdisc.com/
  - codezero 10y ago
    I had about 1000 DVDs in storage of backed up satellite data. At one point I had to restore the backup using the DVDs and it was about ~8 years into the mission so I was worried that the data would be no good on the earliest discs. It turned out to be totally fine, as far as I could tell. None of the early data had SHA/MD5 sums (I added this later), so there's no guarantee the earlier discs weren't somewhat corrupted, but they were definitely readable. Back then, I had assumed they wouldn't have that long of a life, but as you seem to have found, and I discovered, once written, optical media seems to have a very long shelf life.
    - ghshephard 10y ago
      Most Optical Media degrades almost immediately, depending on storage environment, lasting as little as two two years before they start to degrade/aren't readable (depending on the dye and substrate). The MDISC is archival format - designed to last a minimum of 100 years when stored under reasonable conditions, and up to 1000 years if stored in archival (cool, dark) environments. They are essential carving physical holes into a stone like substrate. The verification tests are really quite impressive: http://www.esystor.com/images/China_Lake_Full_Report.pdf http://www.esystor.com/images/China_Lake_Full_Report.pdf
- protomyth 10y ago
  An LTO drive is good long term since it's guaranteed to write 1 generation back and read 2 generations back. Very easy to use once you get the routine down. As always, remember to do a restore to prove your backup is an actual backup.
  - ekianjo 10y ago
    Thanks. I checked the prices and a LTO drive is... expensive. I would have thought a tape recording device would be cheap by now.
    - protomyth 10y ago
      Nope, but they do last a long time compared to other drives. We have stuff that got retired after 10 years (still working, we just upgraded standards). LTO-7 is out, so you might start picking up deals on older models.
- ars 10y ago
  Store both the tape and the device to read it. PCI/IDE/SATA interfaces are so ubiquitous that even decades from now I fully expect you to be able to use them. If nothing else, via some kind of adapter.
  - varjag 10y ago
    Decades is optimistic for a tape reader or any electronics really. Caps dry out, rubber parts degrade and RoHS solder doesn't help the life expectancy. It might still work, it might not.
- tw04 10y ago
  LTO is the industry standard. You'll have no issues finding an LTO tape drive 10 years from now on ebay, and if for some reason you did, Kroll will still be able to recover it.
codezero 10y ago
I'd like to see some solid data on this. I used to think that DVD+/-R and DVD+/-RW had a shelf life of <10 years (this was ~10 years ago) and I can't recall anyone asserting they lasted longer. After running a long term program where backups were written to DVD+RW discs, and seeing them last > 10 years, I was impressed. Now I'm finding sources that say that the post-write self-life of optical media is on the order of 10s to 100s of years. Anyhow, it just makes me wonder what the practical shelf-life is, and what the underlying media "does" over time :)
- tw04 10y ago
  It breaks down like any material. The easiest way to sample this would be the REALLY cheap cd-r discs you could get in the late 90s. If you left one of those sitting on your dash, they would borderline melt in the sun... and for sure start peeling off the plastic if they sat on your dash for a couple hours. Same thing with newer media - if you keep it in a climate controlled environment the sky is the limit. If you abuse it, it's not going to see anywhere near the quoted life.
- digi_owl 10y ago
  Iirc, RWs las longer than plain Rs. Something to do with there being a second layers between the stprage surface and the environment.
- Amezarak 10y ago
  On the other hand, I've never gotten writable optical media to last longer than three years, despite storing them in a cool dark closet. Did you compare checksums on the data you wrote?
  - codezero 10y ago
    On the earliest data, nope! Very likely I had some or many flipped bits. I did have the backups on multiple discs and comparing the two they had the same data so I guess that's almost as good.

tangue 10y ago

Anecdotal evidence : As an amateur photographer SD cards are the least reliable storage. ( they are flash memory after all...)

codezero 10y ago
How did this manifest? SD cards that just weren't recognized over time? Do you have any other details? Did you stick to expensive brand name SD cards? Did you write to them often, or read from them often? Super curious what the usage pattern was.
- tangue 10y ago
  Yes they're not recognised after a while. To avoid mechanical failures I'm unloading my files straight from my camera via USB and erase the card from the menu. Here's some things I've noticed - They failed more often when I'm shooting video in full HD or 4k (less than a year). When I'm shooting films I can erase a card 6-8 times in a weekend. - For Photos I have cards that last for two-three years, shooting in Raw and erasing the card approx. twice a month. - At first I tried Samsung Pro but it fails equally. I'm using Transcend Extreme Pro but in terms of reliability they're the same things than the budget Transcend imho (the speed is better that's the only reason why I buy them). I had bad experiences with Sandisk cards

Eire_Banshee 10y ago

Depends on how often you write to it

wmf 10y ago

SSDs are only rated for one year of retention with power off; I would expect SD cards to be worse. http://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention http://www.anandtech.com/show/9248/the-truth-about-ssd-data-... In theory flash can be use for archival but it would have to be refreshed periodically.

digi_owl 10y ago
Yeiks. I wonder how many will be burned by that...
SmellyGeekBoy 10y ago
FWIW, I found 4 old SD cards (16MB - 256MB, various manufacturers) in my attic last week that definitely haven't been used in 5 or more years, and the photos on them all copied over to my PC just fine. I'm guessing the much lower density of the smaller capacity cards helps a lot in this regard.

userbinator 10y ago

1TB (8Tbit) sounds almost certainly like it's going to be at least 2-bit if not 3-bit MLC, maybe even 4-bit. The endurance and retention numbers for 3-bit MLC are already pretty dismal in comparison to even 2-bit MLC, and the only way they've been able to convince consumers to buy this stuff is by increasing the capacity and multiplying by it. The cells are still intrinsically more fragile, and increasing the bits-per-cell increases capacity only multiplicatively while retention and endurance decrease exponentially; theoretically, to a very rough approximation, 4 bits per cell will have 4 times the capacity but only 1/16th the endurance and retention of 1 bit per cell. In practice, it's somewhat worse. Instead of a 1TB card using 4bpc flash that may offer 1K cycles per cell (1PB total writes), I'd rather have a 256GB SLC (1bpc, "old-school" flash) card with 16K endurance (4PB total writes). Both take up the same area (thus theoretically cost) and the SLC is in fact simpler in many ways to use because it doesn't really need such elaborate "management", but unfortunately in practice SLC is priced astronomically higher than the technology would suggest. I'm not much for conspiracy theories, but it sometimes makes me really think if the industry is deliberately misleading us into the worse choice in terms of reliability, just so they can sell more. This is one of my big gripes about the flash memory industry --- there is a lot of marketing, a lot of fluff. But if you look at the facts it's clear that no one considering the tradeoffs logically would've ever thought MLC to be in any way better. It can certainly be two, three, or four times the density of SLC, but it doesn't even last a half, third, or fourth as long.

wmf 10y ago
MLC lasts long enough and it is significantly cheaper. That's very logical.
- userbinator 10y ago
  The problem is "long enough" is not... and the manufacturers are continually trying to make that shorter and try to convince you it's "long enough". IMHO a few years is NOT "long enough" for a nonvolatile storage device. A few decades is more reasonable.
keshet 10y ago