5 ms·

This problem seems very overblown to me. The simple answer is "don't skimp." - Use a power supply that can deliver enough current - Use good USB cables that

This problem seems very overblown to me. The simple answer is "don't skimp." - Use a power supply that can deliver enough current - Use good USB cables that can actually carry said current (a lot of people miss this one) - Do not use cheap SD cards - Oversize your card so that the card can properly perform wear-levelling. I have a pair of Raspberry Pi's (2's and/or 3's IIRC) running constantly since 2015. They're running an ancient version of Redsleeve (and are on my list to rebuild but never mind that). DNS, DHCP, and a few other services run on them. They've also survived multiple abrupt powerfail events. They log to the SD card, and yet I've never had a problem. I personally believe that the fact that I'm using oversized cards with gigabytes of free space is the key. Well, perhaps that and the fact that my personal home directory is on NFS. Or maybe I'm just lucky? Who knows...

sdenton4 3y ago
Not sure how things are these days, but it used to be the case that power fluctuations could kill an rpi's sd card. So yeah, you could mitigate with a great power supply and a good surge protector, but in practice I found it hard to figure out which power supplies to actually trust. For my own deployments, I ended up doing read only boot from SD card, making use of a ram disk for temporary files, and hosting the actual os on a USB stick, which was much better shielded from power issues. Worked great, never had issues, but the setup was pretty non standard.
- walteweiss 3y ago
  Chances you have any online notes on that?
  - sdenton4 3y ago
    Sadly, I don't think I do; I was mostly messing around with pi's ten years ago, and don't seem to have blog entries specifically on the usb disk. The main idea, though, was to set up the os so that the boot folder was on the SD card, with root mounted from the usb drive, and /tmp on a ramfs. Iirc, I would install normally on the SD card, and then copy data to the usb stick and edit the boot files to point where they needed to. And finally, set the SD card to read only mode once everything was settled. I think there's better options for all of this on more recent pi's, though. I think you can just boot directly from USB now, for example, instead of relying on the SD card.
magicalhippo 3y ago
Similar. I've been running multiple Pi's for years, and had issues with two. Got a USB cable tester and found both had cables with 1+ Ohms resistance, which would cause drop to less than 4V under load! Swapped both cables to some good ones and never had an issue since. I always use A1 or ideally A2 rated cards from reputable brands bought from reputable stores. Not sure if they help with longevity but the additional IOPS are very noticeable.
Larrikin 3y ago
As someone who kept my Pi 1 in a drawer for years and then bought a Pi 4 after they became available again but after building out a bit of a home lab, the internet consensus seems to be you're lucky. The Pis are energy efficient and worth running apps where catastrophic data loss aren't a concern. I use my Pi 1 with DietPi to run AdGuard and my Pi 4 for simple web apps, like dashboards. But the world has moved forward and there are options like the ODroid that are almost as cheap and energy efficient, without the relying on an SD card. I run my Home Assistant off my ODroid with nvme. They are all backed up to a couple different places, but my failure expectations are completely different. I fully expect my Pis to just have completely ruined cards one day, whereas I am more worried about a bad update to my Home Assistant server.
- walteweiss 3y ago
  Why had you preferred AdGuard over PiHole? Just curious, didn’t follow AdGuard at all, never used it.
  - Larrikin 3y ago
    I looked at both, both seemed like mature products that do exactly the same thing but AdGuard had the better looking UI to me. It worked exactly like I expected so never had a reason to go back and try PiHole.
m463 3y ago
You can also use a usb flash drive for stuff that has to be more reliable, I use openwrt (pi + ntp + usb gps + rtc => a standalone ntp server)
danhor 3y ago
The last few times I used a read-only rootfs it was to build an appliance-like device which you plug in, do stuff and when its not needed can be unplugged. For this, there's no other solution.
zikduruqe 3y ago
I've been running a public facing website on my RPi 2 since 2014 and I am still on the original SD card. But, all assets and logs are being written to tmpfs, and the only thing on the SD card (outside of the OS) is the DB. Or maybe I'm just lucky also.
liotier 3y ago
> The simple answer is "don't skimp" Yes, but then the budget then quickly rises to a point where a compact desktop such as a used Intel NUC is cheaper, on top of being far more powerful.
- Teckla 3y ago
  Depending on how the machine is being used, the Pi could be cheaper through lower electricity costs, right?
  - liotier 3y ago
    If one Pi is sufficient for the task, then nothing will be more energy-efficient. With workloads requiring multiple Pi, the used NUC wins. Your electricity costs may vary...
fragmede 3y ago
While they didn't use tried and proven method that's in raspi-config, it's not about skimping, it's a fundamental underlying change. Your expensive SD will eventually fail. It just will take longer than the cheap one. The question is, when will that be? If you use a read only technique, you don't have to guess.
- speleding 3y ago
  True enough, but if something breaks with a read-only SD you are relying on logging to a cloud service. So you reduce the chance of SD failure but introduce a risk of network failure. Of course, network failure doesn't have to be fatal but you won't have any logs to know what happened. In my experience the chance of one of my Pi's not being able to access the network for one reason or another is much larger than a failed SD card, so I keep my logs on the card.
matja 3y ago
> Oversize your card so that the card can properly perform wear-levelling I think this is half of the problem: SDHC does not require wear-leveling at all, I am unsure of SDXC. But regardless, the wear-levelling algorithm the SD card firmware does is completely invisible, from some vendors it might do a decent job, from some it might be a no-op, we have no way to tell. The only way to know you're getting wear-levelling is to do it at an OS level, e.g. UBIFS stacked on MTD emulation - but there's a bunch of assumptions there which make it unsuitable for SD cards.
willyt 3y ago
Likewise I have 6 Rpi doing various things all with minimum of 64gb sd cards and official Rpi power supply. Oldest running continuously since ~2014. Never had an issue. I just spend £18 on a 128gb sandisk card from a trusted seller instead of ~£4 on an small cheap one.
meitham 3y ago
I too have been worrying about SD card failures in my raspberries but haven’t had a single failure ever! My cards are read/write all Debian and have been running since 2016. I hope this post won’t jinx it but my entire data centre has been running with zero storage failures despite running in the most humid and dusty basement you can imagine, though I did have other hw failures such as power supplies, rams, and ups.

sdenton4 3y ago

Not sure how things are these days, but it used to be the case that power fluctuations could kill an rpi's sd card. So yeah, you could mitigate with a great power supply and a good surge protector, but in practice I found it hard to figure out which power supplies to actually trust. For my own deployments, I ended up doing read only boot from SD card, making use of a ram disk for temporary files, and hosting the actual os on a USB stick, which was much better shielded from power issues. Worked great, never had issues, but the setup was pretty non standard.

walteweiss 3y ago
Chances you have any online notes on that?
- sdenton4 3y ago
  Sadly, I don't think I do; I was mostly messing around with pi's ten years ago, and don't seem to have blog entries specifically on the usb disk. The main idea, though, was to set up the os so that the boot folder was on the SD card, with root mounted from the usb drive, and /tmp on a ramfs. Iirc, I would install normally on the SD card, and then copy data to the usb stick and edit the boot files to point where they needed to. And finally, set the SD card to read only mode once everything was settled. I think there's better options for all of this on more recent pi's, though. I think you can just boot directly from USB now, for example, instead of relying on the SD card.

magicalhippo 3y ago

Similar. I've been running multiple Pi's for years, and had issues with two. Got a USB cable tester and found both had cables with 1+ Ohms resistance, which would cause drop to less than 4V under load! Swapped both cables to some good ones and never had an issue since. I always use A1 or ideally A2 rated cards from reputable brands bought from reputable stores. Not sure if they help with longevity but the additional IOPS are very noticeable.

Larrikin 3y ago

As someone who kept my Pi 1 in a drawer for years and then bought a Pi 4 after they became available again but after building out a bit of a home lab, the internet consensus seems to be you're lucky. The Pis are energy efficient and worth running apps where catastrophic data loss aren't a concern. I use my Pi 1 with DietPi to run AdGuard and my Pi 4 for simple web apps, like dashboards. But the world has moved forward and there are options like the ODroid that are almost as cheap and energy efficient, without the relying on an SD card. I run my Home Assistant off my ODroid with nvme. They are all backed up to a couple different places, but my failure expectations are completely different. I fully expect my Pis to just have completely ruined cards one day, whereas I am more worried about a bad update to my Home Assistant server.

walteweiss 3y ago
Why had you preferred AdGuard over PiHole? Just curious, didn’t follow AdGuard at all, never used it.
- Larrikin 3y ago
  I looked at both, both seemed like mature products that do exactly the same thing but AdGuard had the better looking UI to me. It worked exactly like I expected so never had a reason to go back and try PiHole.

m463 3y ago

You can also use a usb flash drive for stuff that has to be more reliable, I use openwrt (pi + ntp + usb gps + rtc => a standalone ntp server)

danhor 3y ago

The last few times I used a read-only rootfs it was to build an appliance-like device which you plug in, do stuff and when its not needed can be unplugged. For this, there's no other solution.

zikduruqe 3y ago

I've been running a public facing website on my RPi 2 since 2014 and I am still on the original SD card. But, all assets and logs are being written to tmpfs, and the only thing on the SD card (outside of the OS) is the DB. Or maybe I'm just lucky also.

liotier 3y ago

> The simple answer is "don't skimp" Yes, but then the budget then quickly rises to a point where a compact desktop such as a used Intel NUC is cheaper, on top of being far more powerful.

Teckla 3y ago
Depending on how the machine is being used, the Pi could be cheaper through lower electricity costs, right?
- liotier 3y ago
  If one Pi is sufficient for the task, then nothing will be more energy-efficient. With workloads requiring multiple Pi, the used NUC wins. Your electricity costs may vary...

fragmede 3y ago

While they didn't use tried and proven method that's in raspi-config, it's not about skimping, it's a fundamental underlying change. Your expensive SD will eventually fail. It just will take longer than the cheap one. The question is, when will that be? If you use a read only technique, you don't have to guess.

speleding 3y ago
True enough, but if something breaks with a read-only SD you are relying on logging to a cloud service. So you reduce the chance of SD failure but introduce a risk of network failure. Of course, network failure doesn't have to be fatal but you won't have any logs to know what happened. In my experience the chance of one of my Pi's not being able to access the network for one reason or another is much larger than a failed SD card, so I keep my logs on the card.

matja 3y ago

> Oversize your card so that the card can properly perform wear-levelling I think this is half of the problem: SDHC does not require wear-leveling at all, I am unsure of SDXC. But regardless, the wear-levelling algorithm the SD card firmware does is completely invisible, from some vendors it might do a decent job, from some it might be a no-op, we have no way to tell. The only way to know you're getting wear-levelling is to do it at an OS level, e.g. UBIFS stacked on MTD emulation - but there's a bunch of assumptions there which make it unsuitable for SD cards.

willyt 3y ago

Likewise I have 6 Rpi doing various things all with minimum of 64gb sd cards and official Rpi power supply. Oldest running continuously since ~2014. Never had an issue. I just spend £18 on a 128gb sandisk card from a trusted seller instead of ~£4 on an small cheap one.

meitham 3y ago

I too have been worrying about SD card failures in my raspberries but haven’t had a single failure ever! My cards are read/write all Debian and have been running since 2016. I hope this post won’t jinx it but my entire data centre has been running with zero storage failures despite running in the most humid and dusty basement you can imagine, though I did have other hw failures such as power supplies, rams, and ups.