12 ms·

I really can’t understand why SD cards are considered applicable as a main storage device for a general-purpose computer. Reliability is the primary bottleneck

I really can’t understand why SD cards are considered applicable as a main storage device for a general-purpose computer. Reliability is the primary bottleneck in everything, and IO speed second. SD cards are the single worst experience I have had in both speed and reliability in any commodity computer stuff for many years.

farisjarrah 7y ago
Size, Cost, Cross Platform Support and Availability.
- AnthonBerg 7y ago
  Those are very appealing features for sure. My point is specifically that SD cards aren’t actually admissible as a solution - specifically in the context as a main storage device as a general-purpose computer - they are not admissible based only on a cost-benefit ratio because they are broken. Like a glass that is big, cheap, holds any liquid, and is right next to you isn’t admissible if it has a hole in the bottom. There’s another shadow factor. Probably that they are perceived as the only available thing? Probably also it is stupid easy for a hardware designer / implementer to tack on an SD card interface.
  - megous 7y ago
    uSD cards are not broken. You just have to not buy from unknown manufacturers that just put some third party parts together and call it done. And if you use quality power supply and wiring, to avoid brownouts, SD cards can be quite reliable. Choosing an appropriate filesystem may also help (like f2fs). I have several multigb PostgreSQL databases (with checksumming enabled) running from uSD cards for years, scraping data from the web hourly. I find uSD cards quite reliable, actually (for rootfs use), in contrast to what's being said online all the time. I also like that it's removable. If it breaks you just pop in another $4 32GB card, restore, and continue where you left off. Compared to soldered on eMMC, that's much more user friendly.
    - AnthonBerg 7y ago
      I did all the correct things and encountered all the failures described online :) It’s heartening to hear that they’re really, actually workable.
jrockway 7y ago
The reason why rPi uses micro SD is that it's the cheapest possible storage solution. You are not buying an industrial-grade workstation for $35. (For those using the rPi in industrial settings, there is the Compute Module which has eMMC. It adds to the cost, though.) I am not really a fan of "embedded" devices that have to be shut down properly. You can never assume that you won't lose power, and I think there are a lot of flaky/bricked rPis as a result of SD not handing power loss well. But it's cheap, and you just reimage the thing and go when your SD card gets corrupted. When I last worked on embedded Linux, our OS image was read-only and the writable storage was literally initialized at boot with "mount /dev/mmc1 /storage || mkfs.ext4 /dev/mmc1". That is how you get a consistent state after power loss. But throwing everything away at boot doesn't fit the Raspberry Pi model of being half-embedded half-workstation. The result is flakiness.
- Isamu 7y ago
  Thank you for bringing my attention to the rPi Compute Module, I hadn't known this existed.
- derefr 7y ago
  > SD not handing power loss well Is it that SD cards themselves don't "handle power loss well", or is it bad assumptions made by the filesystems chosen to be used on them? Could a filesystem with a specific design avoid SD card corruption?
  - aidenn0 7y ago
    Partial out-of-order writes can happen with an SD card. Most filesystems do not handle partial writes well, particularly when they happen out-of-order. A redundant checksumming filesystem like ZFS might be better, but that's only because ZFS can totally handle up to N drives per vdev going totally batshit (where N is configurable when setting up the vdev), not because it is designed for SD cards. Spinning drives can also have misdirected writes, but enterprise drives have for decades now (older than SATA) supported commands for flushing their caches, so proper barriers can be maintained.
  - jrockway 7y ago
magicalhippo 7y ago
So what is it that makes SD cards unreliable for use-cases such as this? Is it just the limited amount of flash available for remapping? If so, can one use the same trick as with regular SSDs and simply avoid partitioning some space to improve reliability?
- deleted 7y ago
  [deleted]
- craftyguy 7y ago
  Not OP, but in my experience it's the broad range of SD card quality on the market, and the sketchy/'temporary' physical connection between the gold contacts on the card and the gold contacts in the card reader. In this case, the reliability issues result in the storage device dropping out sporadically entirely during reads and/or writes, or dramatically varying speeds sometimes as low as several KB/s.
pkaye 7y ago
Speaking from SSD firmware development experience, the reason is those devices are so hardware constrained (in particular SRAM and code) that the firmware team spends a lot of time being clever in making it all work under huge time pressure. Couple that with SD cards being a testbed for the newest flash technology. That is why I don't use SD card for anything I care about reliability.
- everybodyknows 7y ago
  Any alternative suggestions for air-gapped home PC backup?
  - pkaye 7y ago
    My primary concern is that those small form factor storage media might not be so reliable for backup. I'd be okay with mSATA , SATA or NVMe form factor from a reputable manufacturer. I always use something one generation behind or been out for atleast an year. Let others test the new NAND and firmware.
  - pjc50 7y ago
    How do you airgap a backup?
    - everybodyknows 7y ago
      One way is to have a pool of two or three removable flash devices, with only one plugged in at a time, the others in safe places elsewhere. Automate nightly date-stamped copy to the card of just your essential data. At some interval that strikes a balance between convenience and safety, cycle the cards. You'll also need a scheme for pruning older backups to recover storage.
      - pkaye 7y ago
        In that case, understand the SSDs these days don't have great retention specs when not powered. Its one of the drawbacks of using smaller transistors and multi-level cell technology to increase capacity. When powered up they can do a background scan to scrub the data though.
mycall 7y ago
I wish rPI could boot from network
- bri3d 7y ago
  The RPi 3 and late revisions of the RPi 2 support PXE on the built-in Ethernet adapter: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net.md https://www.raspberrypi.org/documentation/hardware/raspberry... The RPi 4 has a boot process with a programmable boot ROM that should allow for a more "real" bootloader like uboot in the long term, but currently PXE isn't supported.
ksaj 7y ago
In a 5 - 55 dollar device, they make a lot of sense. If your application is such that SD isn't good enough, the rpi itself probably isn't either. Both are great for the price to capability ratio. Of course they won't last forever, but you don't complain about not getting ceramic mugs at the local takeout coffee joint, either.

farisjarrah 7y ago

Size, Cost, Cross Platform Support and Availability.

AnthonBerg 7y ago
Those are very appealing features for sure. My point is specifically that SD cards aren’t actually admissible as a solution - specifically in the context as a main storage device as a general-purpose computer - they are not admissible based only on a cost-benefit ratio because they are broken. Like a glass that is big, cheap, holds any liquid, and is right next to you isn’t admissible if it has a hole in the bottom. There’s another shadow factor. Probably that they are perceived as the only available thing? Probably also it is stupid easy for a hardware designer / implementer to tack on an SD card interface.
- megous 7y ago
  uSD cards are not broken. You just have to not buy from unknown manufacturers that just put some third party parts together and call it done. And if you use quality power supply and wiring, to avoid brownouts, SD cards can be quite reliable. Choosing an appropriate filesystem may also help (like f2fs). I have several multigb PostgreSQL databases (with checksumming enabled) running from uSD cards for years, scraping data from the web hourly. I find uSD cards quite reliable, actually (for rootfs use), in contrast to what's being said online all the time. I also like that it's removable. If it breaks you just pop in another $4 32GB card, restore, and continue where you left off. Compared to soldered on eMMC, that's much more user friendly.
  - AnthonBerg 7y ago
    I did all the correct things and encountered all the failures described online :) It’s heartening to hear that they’re really, actually workable.

jrockway 7y ago

The reason why rPi uses micro SD is that it's the cheapest possible storage solution. You are not buying an industrial-grade workstation for $35. (For those using the rPi in industrial settings, there is the Compute Module which has eMMC. It adds to the cost, though.) I am not really a fan of "embedded" devices that have to be shut down properly. You can never assume that you won't lose power, and I think there are a lot of flaky/bricked rPis as a result of SD not handing power loss well. But it's cheap, and you just reimage the thing and go when your SD card gets corrupted. When I last worked on embedded Linux, our OS image was read-only and the writable storage was literally initialized at boot with "mount /dev/mmc1 /storage || mkfs.ext4 /dev/mmc1". That is how you get a consistent state after power loss. But throwing everything away at boot doesn't fit the Raspberry Pi model of being half-embedded half-workstation. The result is flakiness.

Isamu 7y ago
Thank you for bringing my attention to the rPi Compute Module, I hadn't known this existed.
derefr 7y ago
> SD not handing power loss well Is it that SD cards themselves don't "handle power loss well", or is it bad assumptions made by the filesystems chosen to be used on them? Could a filesystem with a specific design avoid SD card corruption?
- aidenn0 7y ago
  Partial out-of-order writes can happen with an SD card. Most filesystems do not handle partial writes well, particularly when they happen out-of-order. A redundant checksumming filesystem like ZFS might be better, but that's only because ZFS can totally handle up to N drives per vdev going totally batshit (where N is configurable when setting up the vdev), not because it is designed for SD cards. Spinning drives can also have misdirected writes, but enterprise drives have for decades now (older than SATA) supported commands for flushing their caches, so proper barriers can be maintained.
- jrockway 7y ago

magicalhippo 7y ago

So what is it that makes SD cards unreliable for use-cases such as this? Is it just the limited amount of flash available for remapping? If so, can one use the same trick as with regular SSDs and simply avoid partitioning some space to improve reliability?

deleted 7y ago
[deleted]
craftyguy 7y ago
Not OP, but in my experience it's the broad range of SD card quality on the market, and the sketchy/'temporary' physical connection between the gold contacts on the card and the gold contacts in the card reader. In this case, the reliability issues result in the storage device dropping out sporadically entirely during reads and/or writes, or dramatically varying speeds sometimes as low as several KB/s.

pkaye 7y ago

Speaking from SSD firmware development experience, the reason is those devices are so hardware constrained (in particular SRAM and code) that the firmware team spends a lot of time being clever in making it all work under huge time pressure. Couple that with SD cards being a testbed for the newest flash technology. That is why I don't use SD card for anything I care about reliability.

everybodyknows 7y ago
Any alternative suggestions for air-gapped home PC backup?
- pkaye 7y ago
  My primary concern is that those small form factor storage media might not be so reliable for backup. I'd be okay with mSATA , SATA or NVMe form factor from a reputable manufacturer. I always use something one generation behind or been out for atleast an year. Let others test the new NAND and firmware.
- pjc50 7y ago
  How do you airgap a backup?
  - everybodyknows 7y ago
    One way is to have a pool of two or three removable flash devices, with only one plugged in at a time, the others in safe places elsewhere. Automate nightly date-stamped copy to the card of just your essential data. At some interval that strikes a balance between convenience and safety, cycle the cards. You'll also need a scheme for pruning older backups to recover storage.
    - pkaye 7y ago
      In that case, understand the SSDs these days don't have great retention specs when not powered. Its one of the drawbacks of using smaller transistors and multi-level cell technology to increase capacity. When powered up they can do a background scan to scrub the data though.

mycall 7y ago

I wish rPI could boot from network

bri3d 7y ago
The RPi 3 and late revisions of the RPi 2 support PXE on the built-in Ethernet adapter: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net.md https://www.raspberrypi.org/documentation/hardware/raspberry... The RPi 4 has a boot process with a programmable boot ROM that should allow for a more "real" bootloader like uboot in the long term, but currently PXE isn't supported.

ksaj 7y ago

In a 5 - 55 dollar device, they make a lot of sense. If your application is such that SD isn't good enough, the rpi itself probably isn't either. Both are great for the price to capability ratio. Of course they won't last forever, but you don't complain about not getting ceramic mugs at the local takeout coffee joint, either.