12 ms·

8Gb USB Flash Drive Endurance Test (2017)

74 points by daxuak 5y ago

ugjka 5y ago
I had an 8Gb drive plugged into my Raspberry pi and it managed to kill it even without any writes happening but I guess I had not disabled the file access time stamp updates
- placatedmayhem 5y ago
  I do wish access times were disabled by default. In my experience, they don't generally provide any useful information, but place unneeded stress on the storage.
  - chasil 5y ago
    Their original purpose was to help administrators identify unused files to migrate to auxiliary storage. This is discussed in a BtrFS post on the atime impact to snapshots (quite harmful). Definitely mount BtrFS with noatime when snapshots are in use (likely best with ZFS also). https://lwn.net/Articles/499293/ https://lwn.net/Articles/499293/
LeoPanthera 5y ago
This is a bit of an aside but when using dd, larger block sizes aren't always faster. On modern systems 512k tends to be optimal, smaller sizes around 128k may be better on older/slower systems. The 8M block size used here is certainly excessive.
- userbinator 5y ago
  Could be an attempt at matching the erase block size of the NAND. Of course due to the FTL and other abstraction layers in between, it probably didn't have the intended effect.
nix23 5y ago
Question, anyone knows a proven industrial grade usb-stick ~16G for server 24/7 usage?
- chasil 5y ago
  Look for "SLC USB flash" on eBay. Definitely confirm the part specs with the OEM, and maybe test one to confirm longevity. https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:whAAAOSwhh5To7Qt https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:...
  - nix23 5y ago
    Hey thanks for the tip, i knew about SLC's but i meant something that is really proven, but also small so you can putt it into pizza-form-factor servers. Something like that: https://www.atpinc.com/de/products/industrial-ssds-usb-drive https://www.atpinc.com/de/products/industrial-ssds-usb-drive But available and more then 2GB ;) EDIT: Nice i found something https://www.swissbit.com/de/produkte/produktsuche/produkte/ https://www.swissbit.com/de/produkte/produktsuche/produkte/
    - chasil 5y ago
      I see this listing for Oracle/Sun SLC media. When you say "really proven," do you mean a lifetime of 100k writes, as is normally expected of SLC devices? p.s. I've bought SwissBit before, and had no problems. https://www.ebay.com/itm/165027448860?hash=item266c67081c:g:7AEAAOSwOYlhHwOv https://www.ebay.com/itm/165027448860?hash=item266c67081c:g:...
      - nix23 5y ago
        >When you say "really proven," I mean something like: We used them for every esxi servers since ~5 years and never had a problem with it. I just have a really hard time trusting any usb-stick. But thanks the Oracle hint/link is a good one.
        5y ago
chasil 5y ago
Heating flash media can repair the oxide layer, depending upon temperature and duration. In 2012, research on short bursts of 800°C heat could vastly extend media lifetime. Supposedly 250° for several hours has the same effect (assuming the packaging can withstand it). In view of the shortages of recent years, we should mandate some form of this technology so we don't waste foundary output on throw-away parts (planned obsolescence). https://m.hexus.net/tech/news/storage/48893-making-flash-memory-reliable-800c-heat-pulses/ https://m.hexus.net/tech/news/storage/48893-making-flash-mem...
- nine_k 5y ago
  At 250°C, all components would desolder from the PCB. Provided that the components are attached at one side, one would need to be very careful while heating and cooling the device.
  - SV_BubbleTime 5y ago
    > Provided that the components are attached at one side Unlikely with the tiny size of these devices. I am sure there are tons, but that most will have components on the A and B side. However… the same reasons that this works to assemble in reflow, are the same reasons it MIGHT not be an issue here. Surface tension keeps the hot upside-down parts where they should be when their solder is molten. So if you have the right conditions, and you have the correct side-up, it should be no more of an issue to reflow a two sided part than a one. This is likely how it was made to begin with, the boards run down the line twice and the second time the upside parts have molten solder. Obviously there would be issues for things touching components like the enclosure or tape/sticker bridging multiple parts. And fwiw to anyone… you determine “which side is up” by the weight of the components. If one side has a heavy part that won’t be held in place by surface tension, that’s the top side. I wouldn’t do any of this for some internet-rumor that your drive will last longer, but I do know enough about two sided reflow to offer the above. Edit: lol classic
  - R0b0t1 5y ago
    If it's not disturbed it could survive this, but ideally you can reduce the temperature just a bit further and increase the time period (though for some processes there is a temperature floor) so you don't liquefy the solder. If you just heat it and don't disturb the surface it should survive, but I'mma have to look up tin oxidation again. It might foam. Would be interesting to have media with a self-heating cycle.
ctoth 5y ago
I'm using the Bar Plus 128GB plugged into my RT-AX58U to host Diversion, a dns-based adblocker and it wasn't until recently that I realized that Dnsmask was writing a log for every domain resolved... Wonder how much of the drive lifetime I burnt through there.
- sgtnoodle 5y ago
  It probably worked out to only a few GB total, so probably not very much?
- riobard 5y ago
  I'm wondering about this for a long time and hope if any experts can explain this: is there any difference in append-only & small-sized log writing vs sequentially filling an SSD/USB flash drive/SD cards? I guess there are significant differences due to flash wear-leveling and write-amplification but I'm not sure. What I know so far is that there's difference in wear-leveling because apparently cheap USB flash drives/SD cards might not do global wear-leveling but rather only limited local wear-leveling. This shouldn't be a problem for append-only logging because it's not writing to the same block repeatedly. However for write-amplification I'm a bit confused. Does append-only logging to an existing NAND block require reprogramming as well? If so, small appends will leave to very large ratio of write-amplification and significantly shorten the NAND life time, right? This is particularly important for devices like Raspberry Pi which rely on USB flash drives/SD cards for logging.
userbinator 5y ago
The last to fail was the Verbatim Store’n’Go which achieved an impressive 9751 cycles. Nice to see --- I have a 64MB Verbatim SnG from many years ago with Samsung SLC that has probably been rewritten many times over, yet only a tiny fraction of the rated 100K cycles has been used, and actually has a full binary size (131072 512-byte sectors are accessible).
techrat 5y ago
The article mentions lifetime/longevity/data retention, although not an 8GB drive that had been written to multiple times... I recently purchased a 2GB Sandisk Cruzer Titanium (nostalgia) thumbdrive off eBay, new in package. Since drives at the time typically had preinstalled software like U3 or some app to encrypt your files, I wondered if a drive as old as this one would still be readable, never having had power since it was packaged up. It was. https://i.imgur.com/0dNo5ni.png https://i.imgur.com/0dNo5ni.png 14 years. It outlived nearly every CDR and DVDR I had.
- userbinator 5y ago
  Retention specs are usually for "end of life" conditions, i.e. after the blocks have been cycled the specified number according to endurance. If they were only written once, and the device kept at room temperature, I would not be surprised if it lasted a century or more --- the retention is an exponential function of both temperature and cycles, if I remember correctly from the last time I researched this stuff in detail.
holoduke 5y ago
I am waiting for my raspberry pi running home assistant on a 32gb MicroSD to die. It's running now for about 3 years nonstop. I am amazed it is still running.
sbassi 5y ago
So what is recommended for long term storage? (not many writes, I mean, for offsite backup).
- n3dm 5y ago
  They make archival quality optical media for all types. Here's some Blu Ray with 1000 yr shelf life. https://www.amazon.com/Archival-Verbatim-M-Disc-Inkjet-Printable/dp/B01B99WUR6/ https://www.amazon.com/Archival-Verbatim-M-Disc-Inkjet-Print...
- robotnikman 5y ago
  Tape drives kept in climate controlled storage, however the upfront cost is expensive

74 points by daxuak 5y ago

I had an 8Gb drive plugged into my Raspberry pi and it managed to kill it even without any writes happening but I guess I had not disabled the file access time stamp updates

placatedmayhem 5y ago
I do wish access times were disabled by default. In my experience, they don't generally provide any useful information, but place unneeded stress on the storage.
- chasil 5y ago
  Their original purpose was to help administrators identify unused files to migrate to auxiliary storage. This is discussed in a BtrFS post on the atime impact to snapshots (quite harmful). Definitely mount BtrFS with noatime when snapshots are in use (likely best with ZFS also). https://lwn.net/Articles/499293/ https://lwn.net/Articles/499293/

LeoPanthera 5y ago

This is a bit of an aside but when using dd, larger block sizes aren't always faster. On modern systems 512k tends to be optimal, smaller sizes around 128k may be better on older/slower systems. The 8M block size used here is certainly excessive.

userbinator 5y ago
Could be an attempt at matching the erase block size of the NAND. Of course due to the FTL and other abstraction layers in between, it probably didn't have the intended effect.

nix23 5y ago

Question, anyone knows a proven industrial grade usb-stick ~16G for server 24/7 usage?

chasil 5y ago
Look for "SLC USB flash" on eBay. Definitely confirm the part specs with the OEM, and maybe test one to confirm longevity. https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:whAAAOSwhh5To7Qt https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:...
- nix23 5y ago
  Hey thanks for the tip, i knew about SLC's but i meant something that is really proven, but also small so you can putt it into pizza-form-factor servers. Something like that: https://www.atpinc.com/de/products/industrial-ssds-usb-drive https://www.atpinc.com/de/products/industrial-ssds-usb-drive But available and more then 2GB ;) EDIT: Nice i found something https://www.swissbit.com/de/produkte/produktsuche/produkte/ https://www.swissbit.com/de/produkte/produktsuche/produkte/
  - chasil 5y ago
    I see this listing for Oracle/Sun SLC media. When you say "really proven," do you mean a lifetime of 100k writes, as is normally expected of SLC devices? p.s. I've bought SwissBit before, and had no problems. https://www.ebay.com/itm/165027448860?hash=item266c67081c:g:7AEAAOSwOYlhHwOv https://www.ebay.com/itm/165027448860?hash=item266c67081c:g:...
    - nix23 5y ago
      >When you say "really proven," I mean something like: We used them for every esxi servers since ~5 years and never had a problem with it. I just have a really hard time trusting any usb-stick. But thanks the Oracle hint/link is a good one.
      - 5y ago

chasil 5y ago

Heating flash media can repair the oxide layer, depending upon temperature and duration. In 2012, research on short bursts of 800°C heat could vastly extend media lifetime. Supposedly 250° for several hours has the same effect (assuming the packaging can withstand it). In view of the shortages of recent years, we should mandate some form of this technology so we don't waste foundary output on throw-away parts (planned obsolescence). https://m.hexus.net/tech/news/storage/48893-making-flash-memory-reliable-800c-heat-pulses/ https://m.hexus.net/tech/news/storage/48893-making-flash-mem...

nine_k 5y ago
At 250°C, all components would desolder from the PCB. Provided that the components are attached at one side, one would need to be very careful while heating and cooling the device.
- SV_BubbleTime 5y ago
  > Provided that the components are attached at one side Unlikely with the tiny size of these devices. I am sure there are tons, but that most will have components on the A and B side. However… the same reasons that this works to assemble in reflow, are the same reasons it MIGHT not be an issue here. Surface tension keeps the hot upside-down parts where they should be when their solder is molten. So if you have the right conditions, and you have the correct side-up, it should be no more of an issue to reflow a two sided part than a one. This is likely how it was made to begin with, the boards run down the line twice and the second time the upside parts have molten solder. Obviously there would be issues for things touching components like the enclosure or tape/sticker bridging multiple parts. And fwiw to anyone… you determine “which side is up” by the weight of the components. If one side has a heavy part that won’t be held in place by surface tension, that’s the top side. I wouldn’t do any of this for some internet-rumor that your drive will last longer, but I do know enough about two sided reflow to offer the above. Edit: lol classic
- R0b0t1 5y ago
  If it's not disturbed it could survive this, but ideally you can reduce the temperature just a bit further and increase the time period (though for some processes there is a temperature floor) so you don't liquefy the solder. If you just heat it and don't disturb the surface it should survive, but I'mma have to look up tin oxidation again. It might foam. Would be interesting to have media with a self-heating cycle.

ctoth 5y ago

I'm using the Bar Plus 128GB plugged into my RT-AX58U to host Diversion, a dns-based adblocker and it wasn't until recently that I realized that Dnsmask was writing a log for every domain resolved... Wonder how much of the drive lifetime I burnt through there.

sgtnoodle 5y ago
It probably worked out to only a few GB total, so probably not very much?
riobard 5y ago
I'm wondering about this for a long time and hope if any experts can explain this: is there any difference in append-only & small-sized log writing vs sequentially filling an SSD/USB flash drive/SD cards? I guess there are significant differences due to flash wear-leveling and write-amplification but I'm not sure. What I know so far is that there's difference in wear-leveling because apparently cheap USB flash drives/SD cards might not do global wear-leveling but rather only limited local wear-leveling. This shouldn't be a problem for append-only logging because it's not writing to the same block repeatedly. However for write-amplification I'm a bit confused. Does append-only logging to an existing NAND block require reprogramming as well? If so, small appends will leave to very large ratio of write-amplification and significantly shorten the NAND life time, right? This is particularly important for devices like Raspberry Pi which rely on USB flash drives/SD cards for logging.

userbinator 5y ago

The last to fail was the Verbatim Store’n’Go which achieved an impressive 9751 cycles. Nice to see --- I have a 64MB Verbatim SnG from many years ago with Samsung SLC that has probably been rewritten many times over, yet only a tiny fraction of the rated 100K cycles has been used, and actually has a full binary size (131072 512-byte sectors are accessible).

techrat 5y ago

The article mentions lifetime/longevity/data retention, although not an 8GB drive that had been written to multiple times... I recently purchased a 2GB Sandisk Cruzer Titanium (nostalgia) thumbdrive off eBay, new in package. Since drives at the time typically had preinstalled software like U3 or some app to encrypt your files, I wondered if a drive as old as this one would still be readable, never having had power since it was packaged up. It was. https://i.imgur.com/0dNo5ni.png https://i.imgur.com/0dNo5ni.png 14 years. It outlived nearly every CDR and DVDR I had.

userbinator 5y ago
Retention specs are usually for "end of life" conditions, i.e. after the blocks have been cycled the specified number according to endurance. If they were only written once, and the device kept at room temperature, I would not be surprised if it lasted a century or more --- the retention is an exponential function of both temperature and cycles, if I remember correctly from the last time I researched this stuff in detail.

holoduke 5y ago

I am waiting for my raspberry pi running home assistant on a 32gb MicroSD to die. It's running now for about 3 years nonstop. I am amazed it is still running.

sbassi 5y ago

So what is recommended for long term storage? (not many writes, I mean, for offsite backup).

n3dm 5y ago
They make archival quality optical media for all types. Here's some Blu Ray with 1000 yr shelf life. https://www.amazon.com/Archival-Verbatim-M-Disc-Inkjet-Printable/dp/B01B99WUR6/ https://www.amazon.com/Archival-Verbatim-M-Disc-Inkjet-Print...
robotnikman 5y ago
Tape drives kept in climate controlled storage, however the upfront cost is expensive