5 ms·

For me, the increased perf is nice, but what I really want is increased longevity. IME, microSD is a poor fit for hosting a R/W filesystem. Every time my Pi b

For me, the increased perf is nice, but what I really want is increased longevity. IME, microSD is a poor fit for hosting a R/W filesystem. Every time my Pi based systems have gone down, it's due to a faulty microSD. I was using well rated microSD cards and they still died relatively quickly(months, not years). I suspect they'd be fine if I was running a read-only rootfs, but with a lot of the applications I run(i.e. HomeAssistant) you need R/W capability.

lostapathy 6y ago
Why on earth would you use the microSD when you were booting off an nvme drive, as is the whole point of the OP?
- jbarberu 6y ago
  I think they're saying it's the longevity, rather than the performance boost, that they're excited about.
  - geerlingguy 6y ago
    That's how I read the grandfather comment too—a 3-4 GB/sec NVMe SSD wastes a lot of performance potential on a Pi, but the main thing that is nicer about even low-end SSDs is they have way better write/rewrite longevity than microSD cards, few of which are design in any way for general computing use. Most microSD cards optimize for only one thing—storing large, sequentially-written video files, then copying them off. Rinse and repeat. Even some of the fancy "100 MB/sec+" microSD cards only get like 1-2 MB/sec in random access patterns.
    - megous 6y ago
      4k block random read on a regular A1 rated uSD card is 8MiB/s. I benchmark 11MiB/s on some of my Sandisks Ultras, depending on manufacturing date. Writes usually don't matter much, unless you use an inappropriate filesystem for a SD card or sync a lot. Longevity is longer for SSDs but it's not that bad for uSD cards either. If you setup your system so that there are no pointless writes all the time, they last 3-5 years, no problem, and probably more. The biggest benefit is that uSD cards are much lower power. And unless the use case is very storage heavy, like a constantly accessed database, or backups, SSD is just a waste of resources.
      - felipelemos 6y ago
        > If you setup your system so that there are no pointless writes all the time, they last 3-5 years, no problem, and probably more. Define pointless writes.
        megous 6y ago
        Anything not related to your use case for the board. Random crap general GNU/Linux distros will do because this is not an issue, on usual HW.
      - erik 6y ago
        Consumer mircoSD cards commonly have firmware bugs. A Raspberry Pi running Linux will consistently trigger these bugs given enough time, even if you mount the card read-only. The nand is fine, but the card will be unreadable until reformatted. You might never see this from a single Pi running at home, but people who deploy large fleets of Pis inevitably run into these problems. Industrial microSD cards aren't perfect, but they are more reliable.
        sgtnoodle 6y ago
        Removing an SD card from its socket during a write operation can actually brick the card enough to no longer work at the block level. Perhaps it's brand dependent, but I'm talking about a well known name brand. It's rare, but I know of a ziplock bag of bricked SD cards that shows it's not too rare. I suspect it occurs when a card loses power while in the middle of shifting data around for wear leveling purposes. Presumably, some internal data structure gets borked in a way that the card's embedded MCU firmware didn't anticipate and can't recover from.
        erik 6y ago
        It doesn't even have to be during a write operation. Nand needs to be periodically refreshed after a read, so the controller can be shuffling data even on a read-only mounted card. I've read speculation that most consumer cards store their controller firmware on the same nand that is used as the general storage, and thus are prone to corrupting their own firmware if something goes wrong while shuffling data. Power loss is definitely a threat to an SD card in a Pi. But I've seen many cards go bad while the system is still running, so it's not the whole story. Weirdly, I've had a number of cards that entered a state where they would be inaccessible under Linux or Mac OS. But they could easily be recovered by a re-format under Windows. No idea why. Perhaps the manufacturers test a lot more thoroughly with Windows?
mikkelam 6y ago
log2ram [1] can help you with reducing the writes of logging at least. It saves logging to ram and then writes it once a day. [1] https://github.com/azlux/log2ram https://github.com/azlux/log2ram
- m463 6y ago
  This is a beautiful little utility. I've been doing stuff like this manually for years, and usually throwing away what gets logged.
- jakemauer 6y ago
  Thank you for sharing this, it's comments like this that make it hard to give up skimming HN. I have two Pi's running on SD cards that this is immediately beneficial for, so thank you.
- Avamander 6y ago
  /tmp and /var/log on tmpfs works as well, though it needs a bit of fiddling.
m463 6y ago
You can boot off of usb and use a better (and faster) flash drive.
- willis936 6y ago
  USB commonly has signal integrity issues. Barring those, the connector can just get jostled and poof there goes storage.
  - justin66 6y ago
    > USB commonly has signal integrity issues. That's simply not true.
    - willis936 6y ago
      Yes, it is. Everything about the USB 2.0 PHY is out of control for a 480 Mbps line rate. Things were tightened up in USB 3.0 just as they were in the move to USB 2.0, but not nearly enough to guarantee low BER in real world scenarios at advertised rates. USB is made to be cheap, not reliable.
      - justin66 6y ago
        I do multi-terabyte backups to USB3-connected devices and networked devices. I'm in the habit of using a merge tool for the backups, and using it again to verify that the backups are correct. One thing I've found is that the incidence of flipped bits is extremely small relative to the amount of data transferred, but real, with both USB3 storage devices and network shares accessed via ethernet. Do they use the same error correction algorithm, perhaps? (I honestly haven't looked into it much) An alternate possibility is that the USB3 and ethernet transport error correction is working fine and it's the devices that are not storing data accurately... that seems less likely to me, but somewhere in there is a SATA bus that also has to do error correction. In any case I don't feel great about a statement like "USB commonly has signal integrity issues" since the way I work wouldn't be possible if it were true. It's not perfect, for sure, but I honestly feel very good about my USB-connected SSD storage relative to so many other things I've used in the past: various optical drives, floppies, SD cards, and some not-great hard drives.
fendy3002 6y ago
Not a pi user. Why not SATA? Is it not supported on pi yet?
- BenjiWiebe 6y ago
  Pi's don't have a SATA controller or SATA ports. NVMe doesn't need a controller; it runs over a PCIe lane.
a_conservative 6y ago
I made some flash-based appliances many years ago. There are some things you can do to reduce the number of writes. Have you tried mounting your root fs with `noatime`? Otherwise reads will generate a write to your device (to update the access time).
bootloop 6y ago
The microSD cards usually just get damaged when writes are happening (and this increases with power fluctuation e.g. when you are powering a witless-adapter, zigbee dongle with the Pi). Very likely if you run something like HomeAssistant. The best way to avoid this (without having a pci-e ssd connected to the Pi) is booting from a read-only FS on the SD card and then mounting a USB connected SSD for writing data. If you want to have a "fake" R/W system take a look at https://yagrebu.net/unix/rpi-overlay.md https://yagrebu.net/unix/rpi-overlay.md which is a R-only FS which writes changes into RAM only. I run my PIs with that.

lostapathy 6y ago

Why on earth would you use the microSD when you were booting off an nvme drive, as is the whole point of the OP?

jbarberu 6y ago
I think they're saying it's the longevity, rather than the performance boost, that they're excited about.
- geerlingguy 6y ago
  That's how I read the grandfather comment too—a 3-4 GB/sec NVMe SSD wastes a lot of performance potential on a Pi, but the main thing that is nicer about even low-end SSDs is they have way better write/rewrite longevity than microSD cards, few of which are design in any way for general computing use. Most microSD cards optimize for only one thing—storing large, sequentially-written video files, then copying them off. Rinse and repeat. Even some of the fancy "100 MB/sec+" microSD cards only get like 1-2 MB/sec in random access patterns.
  - megous 6y ago
    4k block random read on a regular A1 rated uSD card is 8MiB/s. I benchmark 11MiB/s on some of my Sandisks Ultras, depending on manufacturing date. Writes usually don't matter much, unless you use an inappropriate filesystem for a SD card or sync a lot. Longevity is longer for SSDs but it's not that bad for uSD cards either. If you setup your system so that there are no pointless writes all the time, they last 3-5 years, no problem, and probably more. The biggest benefit is that uSD cards are much lower power. And unless the use case is very storage heavy, like a constantly accessed database, or backups, SSD is just a waste of resources.
    - felipelemos 6y ago
      > If you setup your system so that there are no pointless writes all the time, they last 3-5 years, no problem, and probably more. Define pointless writes.
      - megous 6y ago
        Anything not related to your use case for the board. Random crap general GNU/Linux distros will do because this is not an issue, on usual HW.
    - erik 6y ago
      Consumer mircoSD cards commonly have firmware bugs. A Raspberry Pi running Linux will consistently trigger these bugs given enough time, even if you mount the card read-only. The nand is fine, but the card will be unreadable until reformatted. You might never see this from a single Pi running at home, but people who deploy large fleets of Pis inevitably run into these problems. Industrial microSD cards aren't perfect, but they are more reliable.
      - sgtnoodle 6y ago
        Removing an SD card from its socket during a write operation can actually brick the card enough to no longer work at the block level. Perhaps it's brand dependent, but I'm talking about a well known name brand. It's rare, but I know of a ziplock bag of bricked SD cards that shows it's not too rare. I suspect it occurs when a card loses power while in the middle of shifting data around for wear leveling purposes. Presumably, some internal data structure gets borked in a way that the card's embedded MCU firmware didn't anticipate and can't recover from.
        erik 6y ago
        It doesn't even have to be during a write operation. Nand needs to be periodically refreshed after a read, so the controller can be shuffling data even on a read-only mounted card. I've read speculation that most consumer cards store their controller firmware on the same nand that is used as the general storage, and thus are prone to corrupting their own firmware if something goes wrong while shuffling data. Power loss is definitely a threat to an SD card in a Pi. But I've seen many cards go bad while the system is still running, so it's not the whole story. Weirdly, I've had a number of cards that entered a state where they would be inaccessible under Linux or Mac OS. But they could easily be recovered by a re-format under Windows. No idea why. Perhaps the manufacturers test a lot more thoroughly with Windows?

mikkelam 6y ago

log2ram [1] can help you with reducing the writes of logging at least. It saves logging to ram and then writes it once a day. [1] https://github.com/azlux/log2ram https://github.com/azlux/log2ram

m463 6y ago
This is a beautiful little utility. I've been doing stuff like this manually for years, and usually throwing away what gets logged.
jakemauer 6y ago
Thank you for sharing this, it's comments like this that make it hard to give up skimming HN. I have two Pi's running on SD cards that this is immediately beneficial for, so thank you.
Avamander 6y ago
/tmp and /var/log on tmpfs works as well, though it needs a bit of fiddling.

m463 6y ago

You can boot off of usb and use a better (and faster) flash drive.

willis936 6y ago
USB commonly has signal integrity issues. Barring those, the connector can just get jostled and poof there goes storage.
- justin66 6y ago
  > USB commonly has signal integrity issues. That's simply not true.
  - willis936 6y ago
    Yes, it is. Everything about the USB 2.0 PHY is out of control for a 480 Mbps line rate. Things were tightened up in USB 3.0 just as they were in the move to USB 2.0, but not nearly enough to guarantee low BER in real world scenarios at advertised rates. USB is made to be cheap, not reliable.
    - justin66 6y ago
      I do multi-terabyte backups to USB3-connected devices and networked devices. I'm in the habit of using a merge tool for the backups, and using it again to verify that the backups are correct. One thing I've found is that the incidence of flipped bits is extremely small relative to the amount of data transferred, but real, with both USB3 storage devices and network shares accessed via ethernet. Do they use the same error correction algorithm, perhaps? (I honestly haven't looked into it much) An alternate possibility is that the USB3 and ethernet transport error correction is working fine and it's the devices that are not storing data accurately... that seems less likely to me, but somewhere in there is a SATA bus that also has to do error correction. In any case I don't feel great about a statement like "USB commonly has signal integrity issues" since the way I work wouldn't be possible if it were true. It's not perfect, for sure, but I honestly feel very good about my USB-connected SSD storage relative to so many other things I've used in the past: various optical drives, floppies, SD cards, and some not-great hard drives.

fendy3002 6y ago

Not a pi user. Why not SATA? Is it not supported on pi yet?

BenjiWiebe 6y ago
Pi's don't have a SATA controller or SATA ports. NVMe doesn't need a controller; it runs over a PCIe lane.

a_conservative 6y ago

I made some flash-based appliances many years ago. There are some things you can do to reduce the number of writes. Have you tried mounting your root fs with `noatime`? Otherwise reads will generate a write to your device (to update the access time).

bootloop 6y ago

The microSD cards usually just get damaged when writes are happening (and this increases with power fluctuation e.g. when you are powering a witless-adapter, zigbee dongle with the Pi). Very likely if you run something like HomeAssistant. The best way to avoid this (without having a pci-e ssd connected to the Pi) is booting from a read-only FS on the SD card and then mounting a USB connected SSD for writing data. If you want to have a "fake" R/W system take a look at https://yagrebu.net/unix/rpi-overlay.md https://yagrebu.net/unix/rpi-overlay.md which is a R-only FS which writes changes into RAM only. I run my PIs with that.