39 ms·

Considerations for a long-running Raspberry Pi

510 points by ilikepi 3y ago

weinzierl 3y ago
As far a I know (and my knowledge inly goes to version 4) the RasPi does not support different power states or sleep modes. It would be interesting how it compares in power consumption to other solutions.
_cenw 3y ago
Alpine in diskless mode and an ethernet cable solve 2/3 of that.
agilob 3y ago
Half of the post is about SD cards, wear, data loss and reliability. Just use SSD?
- teekert 3y ago
  I was expecting that to be tip #1 indeed. And it’s why I went to a Nuc eventually. Choose a nice ssd and it will really run for years and years.
  - yjftsjthsd-h 3y ago
    Nucs also tend to win on performance, software compatibility/upkeep (yay commodity x86), price per performance, and often price outright (pis aren't really $35 if you get a newer/better model and once you buy storage/power/case). AFAICT pis only consistently win on power consumption, and that only just (x86 can get well under 10W, you just have to get the right model)
    - arlecks 3y ago
      Do you have recommendations for a NUC to replace a raspberry pi that draws around 10W when idle?
      - johnchristopher 3y ago
        And possibly one as silent as a fanless pi4 ?
      - enioarda 3y ago
        Have a look at the Beelink SER5, idle around 5W and its quite affordable https://www.notebookcheck.net/Ryzen-5-5560U-performance-debut-Beelink-SER5-mini-PC-review.762934.0.html https://www.notebookcheck.net/Ryzen-5-5560U-performance-debu...
      - dannyw 3y ago
        The Intel N95 and N100 NUCs are amazingly efficient.
  - CTDOCodebases 3y ago
    Thin clients are also an option. Similar power consumption but power and case/cooling/storage is included. Depending on what you choose you get eMMC or SATA or M.2 SATA or M.2 NVMe for storage. For anyone that is interested: https://www.parkytowers.me.uk/thin/ https://www.parkytowers.me.uk/thin/
- srvaroa 3y ago
  Maybe professional bias but I prefer to follow a cattle vs pets approach here and treat them as disposable, meaning: let the sd card break and optimize for quick replacement.
  - alpaca128 3y ago
    Current Pi 4s & 5s support booting from USB out of the box with no configuration (took them long enough), so I don't think it's worth the downtime and wasted SD cards.
  - imtringued 3y ago
    NVMe SSDs cost the same as SD cards nowadays. You can treat the SSDs as disposable if you want, but you are going to quickly realize that this mindset isn't exactly actionable, because there is nothing that needs disposing of.
- imtringued 3y ago
  Before I opened the article I was thinking "Don't use an SD card. Don't use an SD card. Don't use an SD card." People don't understand that 90% of the SD card problems are power related (only relevant if you don't use the official power supply) and 10% are simply because of poor SD card quality. People haven't gotten the message that a charger has lower QC standards since an interruption of the charging process does not shut the device down. A bug that leads to a few milliseconds of power loss will pass QC, but also corrupt your SD card.
  - dboreham 3y ago
    Penny drops. Yes this jives much better with the observed data -- that Rpi break SD cards orders of magnitude more than anything else that uses SD cards no matter how crappy said cards are. Problem could have been solved by adding a decent sized capacitor on the 5v supply rail.
    - justin66 3y ago
      > that Rpi break SD cards orders of magnitude more than anything else that uses SD cards I think if you take into account how often they're powered on you'd find cameras destroy flash memory cards comparably to Raspberry Pis. > Problem could have been solved by adding a decent sized capacitor on the 5v supply rail. Or you could just buy the Raspberry Pi branded power supply.
  - pmontra 3y ago
    Are SSDs less prone to corruption than SD cards when power drops for those milliseconds?
- TaylorAlexander 3y ago
  You can also use a Raspberry Pi compute module in an appropriate carrier. I use the CM4 for work and consequently it’s all I use at home too. Lovely versatile device that fits in whatever carrier you want (including one that mocks a regular pi) and it has industrial grade flash.
  - youngtaff 3y ago
    Do you have any recommendations for a carrier?
    - johnchristopher 3y ago
      What's a carrier in that context ?
      - bemusedthrow75 3y ago
        An IO board that adds the IO ports that the e.g. CM4 lacks. There's a decent range of them from different suppliers, but Raspberry Pi have one themselves called "Raspberry Pi Compute Module 4 IO Board".
        johnchristopher 3y ago
        Thanks !
    - TaylorAlexander 3y ago
      I use this minimal one. Check out what Waveshare has to offer too. https://sourcekit.cc/#/ https://sourcekit.cc/#/
      - youngtaff 3y ago
        Thanks… my ideal would be a similar sized carrier with PoE and M2 support Think I really want something like the Uptime Labs Blade https://pipci.jeffgeerling.com/boards_cm https://pipci.jeffgeerling.com/boards_cm
demomode 3y ago
Also: consider using overlayfs to make root fs read-only
- CTDOCodebases 3y ago
  They mentioned making the root fs read-only which (if they are using the config tools) does use overlayfs: https://raspberrypi.stackexchange.com/questions/124628/raspbian-enable-disable-overlayfs-from-terminal https://raspberrypi.stackexchange.com/questions/124628/raspb...
- devishankar 3y ago
  [flagged]
yjftsjthsd-h 3y ago
I'm shocked the SD card bit isn't first, and more surprised that the post doesn't suggest USB boot (I have one pi that's been on ~24/7 for years now, and I attribute its lack of problems to 1. using Alpine configured to barely touch disk, and 2. not having an SD card to corrupt - I don't know why USB would be more reliable, but anecdotally it is)
- Tor3 3y ago
  I've had a bunch of Pi cards, running on SD without problems. But a single one suddenly developed a super-hot SD card, this was a brand new Pi which I was just setting up. Got the card out, and that one and the next Pi got a USB SSD and are now using those. That was a bit scary. But as mentioned I've also been running Pi with micro SD as Cups servers for years, with no problems at all.
  - Aerbil313 3y ago
    Would there be any reliability problems with using USB flash drives instead of USB SSDs?
    - Tor3 3y ago
      I don't really know. I assume they would be slower than SSDs though, but I have never measured USB flash. The SSBs get some 300MB/sec on my Pi 4 boards.
    - NortySpock 3y ago
      Sample size of 1, but I ran an RPi4 just fine on a USB thumb drive for about a year. Still upgraded to an SSD later because I wanted even more storage space (and SSD seemed to have better random IOPS than a thumb-drive), but I'd say go for it.
- breton 3y ago
  I agree with this completely. All my rpi failures were because of SD cards. I have 2 rpis, both boot and run from usb, both for several years now.
goodburb 3y ago
I'm still using a Sandisk 8GB microSD from 2008 running 24/7: smartphone expansion 6 years -> Orange Pi as a router for 3 years, Pi 4 running Frigate for 3 years. I'm guessing it's SLC/MLC. Had a Transcend 32GB in 2016 die after a year. The biggest issues with set-and-forget setups is software upgrade for security or other reasons, jumping major versions ends up breaking things. Compared to cloud which is (usually) regularly updated.
- adamm255 3y ago
  All my SD cards are Sandisk in all my Pis. (2x B+ and a 4, and a Zero). Never a single issue in over 10 years of running at least 1 Pi 24/7. Only buy Sandisk I guess!
niederman 3y ago
Admittedly, I only have one long-running Raspberry Pi, but it's currently sitting at a few months uptime. And that was an intentional reboot. I've never had to take any measures like these in the four or so years I've had it up.
guenthert 3y ago
No mentioning of flashybrid? I'd thought that's the obvious solution to SD wear (or rather the danger of SD corruption on sudden power loss).
- guenthert 3y ago
  Too late to edit, but I meant to refer to the Debian package of that name, not an actual flash/spinning rust hybrid storage device.
vegabook 3y ago
a) Cable ethernet b) SSD (via USB3.0 adapter on my RPI4) c) Ubuntu Server LTS 22.04. d) cheap UPS. Mine runs Yggdrasil network, HAproxy, Caddy server, a couple of webservers in containers, and a TMUX instance that I log into almost daily to write code (slow computer reveals bad code much better). Since I put it (and my router) on the UPS, in the last 2 years it has literally never gone offline other than a couple of times I rebooted it for firmware upgrades.
- MOARDONGZPLZ 3y ago
  This is mostly what I do for “critical” Pi things. The cable Ethernet and the SSD are major. Do you have any recommendations for a cheap UPS? What are your considerations here?
  - vegabook 3y ago
    I went with a CyberPower BR1200 which was about 150 pounds (circa 200 USD) but that's overkill because it will keep my pi+router going for several hours, whereas power-downs typically last less than 15 minutes max where I live and I haven't seen > 1 hour in 10 years. It's usually "human error" related someone plugs the vacuum in or some appliance trips a circuit. Keeping the router on the UPS (and fiber box in my case) is important though because then everything is completely uninterrupted from a serving perspective. I also give the standard RPI power adapter enough headroom by not overclocking the pi, because in the past this has caused problems when SSD attached with lots of writing, and "big" (for a pi) compute load at the same time. Since turning off the overclocking, zero problems. Probably an RPI5-class power supply would be even better. No clue if this matters but on a jetson nano (which was notoriously power-spikey), crucial (micron) or samsung SSDs tended to be better than budget alternatives.
akira2501 3y ago
I got one of those super low profile USB thumb drives and then set the Pi to boot from that. It ran automation here in my house for 2.5 years without a blip.
guenthert 3y ago
Also no mentioning of the power connector? I have too little experience with USB-C, but the micro USB connector used on early Raspberry Pi's is just asking for trouble. That might be (barely) good enough for charging, but a computing device w/o battery-backup won't take lightly the power interruptions when jiggling the cable a little. I finally got around to replace it with an old-fashioned (time tested!) barrel connector. Easy way to improve the robustness significantly.
- bemusedthrow75 3y ago
  As with my DSLRs/mirrorless cameras (particularly the ones that use the wretched USB mini) when tethering I have ended up with a right-angle adapter cable secured with a cable tie (or the super-strong Tethertools Jerkstopper), and a USB micro cord into that. USB micro is designed to snap at the connector, not the board. And the connectors will indeed snap.
FerretFred 3y ago
I've been running a bunch of Pi's for years now, and the biggest problem I've had is the Pi itself dying: 24/7 usage is hard on a small device. I've also found that stable power is essential, and to that end I've always used 5v 3a branded power cubes, plugged into a pure sine wave UPS. Choice of micro-SDHC cards is important and I ended up getting ATP industrial cards (https://www.atpinc.com/products/industrial-sd-cards https://www.atpinc.com/products/industrial-sd-cards) - expensive but really long-lived. Finally, using RPi-clone (https://github.com/billw2/rpi-clone https://github.com/billw2/rpi-clone) on a regular basis has been a life-saver. I clone to Sandisk Extreme micro-SDHCs and can recover from an outage in minutes.
- vietvu 3y ago
  And how frequent you write to SD card too. My Pi 3B+ have been running for 3 years. Haven't considered to upgrade since my need is small.
rcarmo 3y ago
I did exactly zero of any of those things and have had some Pis run for multiple years without any issues until being replaces by a newer model (my HomeKit/Zigbee gateway and data logger is now a Pi 4). I guess it all boils down to good SD cards and stable power supplies.
- Tor3 3y ago
  Same here - I've been running a couple of Pi3 as Cups servers for years of uptime (the only time that uptime gets reset is when there's a power outage - and that's very rare indeed). Did nothing more than install Raspbian on a micro SD card, set up Cups, connect USB to printer (for one of them - the other manages a networked printer). And left them alone after that.
- dgellow 3y ago
  My Raspberry Pi 2 ran for a while, used as a pihole, a VLC-hacked media center, and a weather stations for simple sensors, until it didn’t :( In ~2022 I started to get random errors, then found out the SD card was failing. I never took the time to fix it, it had lots of little things locally I don’t feel like doing again.
  - domh 3y ago
    It may seem like overkill, but I learnt how to use ansible to manage the software on my pi. I've actually just upgraded my rpi4 to use an ssd so am choosing to start from scratch. I'm hoping my ansible playbooks are written well.
    - dgellow 3y ago
      Haha, yeah, that’s a good point. I may still have some ansible playbook from a while ago but I’m pretty sure it wouldn’t be up to date. I guess it would be nice to have a system like CoreOS was, where you just provide a setup script, systemd unit files, and at runtime most of the file system is expected to be read-only. That way you’re confident you keep all your setup in a git repo and on reboot the whole thing is reset (outside of data stores). It’s just so tempting to quickly ssh into the machine to hack something around, then you forget about it b cause it ~works. But a rpi4 can run containers, so that may also be an alternative.
hcfman 3y ago
My sound localizing Raspberry Pi installs a resilient base system as part of its install. https://github.com/hcfman/sbts-aru https://github.com/hcfman/sbts-aru https://hackaday.com/2023/12/30/localizing-fireworks-launches-with-a-raspberry-pi/ https://hackaday.com/2023/12/30/localizing-fireworks-launche... With one command it for all Pi’s for both Raspbian and bookworm it: * Shrinks the file system (Gee, how does it do that with just one disk ? ;-) ) * Creates new partitions * Installs a memory overlayFS * Installs and configures the system as an audio recorder with micro second time accuracy * updates /etc/rc to do a forced repair of the data and config portions, in case they were damaged. This avoids system hangs waiting for human interaction with fsck For the partitioning scheme it creates a swap partition, not as a wow but as an enabler if you really need it to install some large software. It creates a small config partition. The idea here is that you keep it read only and remount it read write if you need to change config then remount it read only again. And finally a data partition, which in this projects case is where the audio files are written. I maintain a version of an overlayFS boot for the Pi but it needs revisiting for bookworm. The easiest way to use do this is to install the sbts-aru and then just don’t use it. Then everything is done for you in one command. And that version works for all Pi’s. I also do this for the Jetson SBCs. But I need to revisit this for the Orin series. I have it working here for myself and friends but need to update the installer. Note, due to kernel behavior changes with Orin the older Pi like overlayFS code will not work. But I solved this and will release it when I release the Orin release of sbts-install soon. I’ve been using memory overlayFS like installs for years for long running Pi systems.
teleforce 3y ago
For the life of me I don't really get it why Raspberry Pi Foundation does not include onboard eMMC or SSD storage in their non-compute modules products. Yes with the new PCIe expansion in the latest RPi 5 you can have external SSD for example, but if you decided to use it for other purposes as well like extra Ethernet port expansion then you cannot use it for booting anymore.
- bemusedthrow75 3y ago
  > For the life of me I don't really get it why Raspberry Pi Foundation does not include onboard eMMC or SSD storage in their non-compute modules products. Cost. It's always cost. If the eMMC or SSD storage is not enough to hold a general purpose OS for all users, then only some users get the value. And if it is big enough, it's put the cost of the machine up above the point at which they feel happiest, when an SD card is perfectly fine for the majority of their target users. Eben Upton is regularly on record talking about how the cost-per-component/users-per-component tradeoff will lead them to avoid adding a component, and has motivated removing some (composite video for example)
- justin66 3y ago
  It's obviously about cost. It's curious that there are people who need this, are aware of the compute module, and still complain about it. Is the the availability of the compute module that's a problem?
Alifatisk 3y ago
Haven’t followed any of these tips, yet I have barely stumbled upon any issued with my long running pi
blacklion 3y ago
First advice is to enable journaling mode on FS. First advice must be to mount FS in read only mode, mount /var in memory and forward al logs to one nide, which may be not RPi but something with proper UPS and nut running. Power loss becomes absolutely bening if your FSes RO or temporary. It is overkill if you have one RPi but author claims that he uses multiple RPis all around a house. Also, good idea to have A/B system partitions and upgrade system with full partition rewrite and changing active one. Thus way your system will have one good system partition in any case, even if new version has fatal bugs, and recovery become trivial. I'm using several small/single board PCs im different roles in such way for 20+ years with great success.
- Avamander 3y ago
  In addition to /var, tmpfs should also be used for /tmp and similar. That should lengthen the SD card's lifetime immensely.
  - diggan 3y ago
    Isn't this (tmpfs for /tmp) the default setup for most OSes, and surely raspberry's homegrown OS too?
    - tyingq 3y ago
      It seems it is not the default for Raspbian.
      - diggan 3y ago
        TIL! One would think if any distribution had it by default, it would have been Raspbian.
        tyingq 3y ago
        Maybe left off since some of the models have very little memory to spare for a ramdisk?
tempaccount1234 3y ago
I have a few long time running Pis - and have been keeping them up for a decade now. No SD Card corruption ever and i got close to 1000 days uptime on one. The biggest problem is loss of wifi, after a few months one will lose wifi, but keep working - it’s constantly recording data so a reboot is not a good idea. I’d prefer a solution where I could just reset the wifi, but all attempts to script that reliably so far failed.
- nottorp 3y ago
  I've got a few in factories (doing non critical stuff) at work. They auto reboot regularly because otherwise they'll lose whatever's on usb (reading those usb metering devices being the whole point of the PIs).
Namidairo 3y ago
I'd consider enabling the hardware watchdog as well. While one could argue that you should figure out the source of your device freezing in the first place; Nothing is better than having to ask someone to power cycle your Raspberry Pi while you're away.
- drpixie 3y ago
  Yes. And I monitor my network link and automatically power-cycle the modem, if the worst happens. (Which is rare, but the network link has been the source of most of the few problems I've had.) And use a wired network connection for anything important!
Temetra 3y ago
I've been plagued with the wifi problem since changing routers. Devices on the local network will randomly lose the ability to connect to it, but everything else is fine.
- forinti 3y ago
  I added a cron job to one Pi that checks if wifi is up and tries to restart it if not. I also had an LG monitor that had so much feedback it would disable the WiFi interface completely. So I would check your monitor, if the pi is connected to one.
borlox 3y ago
Just read the data sheet of the Sandisk Max Endurance and, oh boy, what a fsck’ing marketing bs language. They state the endurance on thousands of hours of FHD video, but what assumptions do they make in bitrate etc? Can‘t they state total TB written or drive writes per day or something sensible?
- userbinator 3y ago
  Flash memory manufacturers are notoriously secretive about the actual endurance of their products, likely because it's now embarrassingly low. Specifying endurance in "thousands of hours of FHD video" implies large sequential writes, or in other words the best-case for write amplification.
  - dannyw 3y ago
    They also substitute parts all the time. Even the “best” brands have been known to do this. From different controllers to swapping TLC for QLC flash.
- Shakahs 3y ago
  Better manufacturers do give you a TBW endurance rating, and document what type of flash is being used (SLC, MLC, etc.). I used one of these Micron cards in my RPi4, it has high write endurance and also an A2 performance rating so it can support the IOPS needs of a boot drive. https://www.mouser.com/new/micron-technology/micron-i400-microsd-cards/ https://www.mouser.com/new/micron-technology/micron-i400-mic...
yokoprime 3y ago
I’ve got 2 rPI’s thats been running for 4+ years each. Only thing I do is update them once in a while. Both have workloads running 24/7. no issues. I have however experienced issues in the past but those were due to a faulty power adapter not keeping voltage within spec.
userbinator 3y ago
It's worth noting that SD card firmware is usually optimised for the FAT32 filesystem, which has a very predictable access (specifically write) pattern, and using filesystems that have a more "free-form" layout can lead to lower performance and higher write amplification: https://lwn.net/Articles/428584/ https://lwn.net/Articles/428584/
- goodburb 3y ago
  Good note, I wonder if wear leveling depends on that since it's local wear leveling compared to SSD global. SDXC probably requires exFAT and free space bitmap is considered.
sdflhasjd 3y ago
This covers the readonly filesystem, but doesn't cover the write protect flag that you can set on the microSD card itself[1]. The flag will configure the card's controller to drop any writes, and is thought to resist the corruption issues that can still occur even when the filesystem is readonly. Also, creating a readonly root out of an existing disto is a bit of a pain, my preference is to use a distro (like TinyCore) that's already a readonly root. https://github.com/BertoldVdb/sdtool https://github.com/BertoldVdb/sdtool
zh3 3y ago
As someone using many, many Pi's at home and many times that at work, the preferred approach is to boot them all diskless (and for those that actually need an SD card, boot them off a read-only SD card and get everything else off the home server). This is so much easier than having lots of different SD cards/Pi versions etc, and makes them trivial to replace in the event of failure.
- isoprophlex 3y ago
  I have several of them, sounds ideal. can you perhaps link me to a good/useful writeup on how to accomplish this?
  - zh3 3y ago
    Look up diskless booting, it's a very general thing on linux (i.e. not Pi-specific, although there are plenty of Pi-specific tutorials on the 'net. As a minimal first step, install an NFS server (which can be X86, Pi, other) on your LAN and make sure you can mount it from the pi ("mount $SERVER:/some/dir /mnt/tmp"). Then copy the contents of a Pi SD card to the server, make it exportable (see '/etc/exports') then edit '/etc/fstab' on the Pi to mount the (now remote) copy of the SD card instead of the usual root. That should get you started - beyond that, with some of the Pi's you don't even need to have an SD card installed (however you'll then need to set up things like DHCP and TFTP on your server).
    - isoprophlex 3y ago
      Thanks for sharing!
    - nsteel 3y ago
      There are also some pi-specific bootloader config options that can make life easier, they are documented at https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#network-booting https://www.raspberrypi.com/documentation/computers/raspberr... and from https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#TFTP_FILE_TIMEOUT https://www.raspberrypi.com/documentation/computers/raspberr... onwards There's also a tutorial at https://www.raspberrypi.com/documentation/computers/remote-access.html#network-boot-your-raspberry-pi https://www.raspberrypi.com/documentation/computers/remote-a...
LeoPanthera 3y ago
All my home Pis network boot, so there is no card to fail. You can also change what OS they boot into by just renaming a symlink on the server and rebooting them. Very convenient.
- jon-wood 3y ago
  I'm curious what you're using all these Pis for when you've got a server on the network to provide network boot. Can you not just use the device that's providing network boot to provide whatever you're running on the Pis?
  - LeoPanthera 3y ago
    No, because the server is in the wrong place. Some Pis are connected to televisions and play videos, or emulated games. One is connected to my hi-fi. One has a GPS receiver on it for NTP. One is a photo frame. Basically, the Pis exist to interface with the real world.
csydas 3y ago
i very much so enjoyed these articles even if i don’t use raspberry pi anymore. it has a very much so research notes cleaned up for sharing feel and reminds me of some of my teams internal articles, particularly on research the author wasn’t sure on but wanted to put out there regardless. it was very easy to get their decisions and the limit of their research and knowledge so i had pretty clear idea what i need to still check on my own and what i might adopt if i were to use rpi devices. i don’t really get what to use a rpi for but i guess not important, it was just a nice series of articles
- rcMgD2BwE72F 3y ago
  I can't do without my https://en.wikipedia.org/wiki/LibreELEC https://en.wikipedia.org/wiki/LibreELEC RPI.
lqet 3y ago
I have 2 Pis running basically non-stop (2-3 power outages) with the same SD card since 2017 (DNS/print server and Kodi, media is on external NFS). The only thing I did was to disable all logs. Never had a single problem. They both have SanDisk 2 GB cards in them. I vaguely remember naively thinking along the lines of "less space => less bit density => better reliability".
- abc123abc123 3y ago
  Amen! Exact same kodi-setup and sandisk cards, and has been happily running for years without problems. Disable all logs, media on smb/nfs, and off you go.
- iforgotpassword 3y ago
  I'm still running Kodi on a pi1 approaching 10 years of runtime on the original SD card. It's powered on most of the time but sometimes I accidentally power of the USB power supply it's connected to. It's a little 5-port one that's about just as old with a power button that's easy to press accidentally in my specific setup.
- zikduruqe 3y ago
  I have a RPi that has been running non-stop continuously since 2014 on the same SD card, serving up a weather website. I basically mounted all logging and webpages on tmpfs, and the DB resides on the SD card being written to every 5 minutes.
  - seb1204 3y ago
    Are there instructions on how to upgrade my pihole to this?
    - zikduruqe 3y ago
      Start here. https://ostechnix.com/how-to-mount-a-temporary-partition-in-ram-in-linux/ https://ostechnix.com/how-to-mount-a-temporary-partition-in-... Just understand that anything that is written to tmpfs will be lost if you have a reboot. It might make troubleshooting difficult if you need to preserve them for whatever reason.
hcfman 3y ago
Another nice thing you can do if you use the multi-partitioned memory overlayFS approach I mentioned earlier is you can make /var/docker be a symlink to your read-write data partition. Obviously you are going to have problems using docker with the standard memory overlayFS approach. Also nice, is you can make your data or other partitions be encrypted. I've done this before. On the Pi 5 you can use the standard encryption as there's hardware support. On earlier Pi's you can use the encryption used for android. This does means there's a manual step in the startup for you to enter your encryption password.
planb 3y ago
Buy a decent SD card and overprovision the space and wear leveling will take care of the rest. I have a Pi with a 128GB SD card and a 32GB filesystem on it running for 6 years straight without problems now. No need to disable logs, just disable debug logging so you won‘t generate gigabytes a day.
hcfman 3y ago
Back in 2011 I made a commercial product that ran on the earliest plug computers from global scale technologies. I only sold 20 of 'em and every single one of them was being returned with SD card corruption problems. I had to quickly pivot to keeping the rootfs read-only. I've been a fan ever since. Incidentally, that early commercial product was a home security product with a very small amount of home automation. I released this into open source with a new name in 2021 and now runs on the Jetson series SBCs (https://github.com/hcfman/sbts-install https://github.com/hcfman/sbts-install). Except then including high end YOLO models as triggers. Because it was intended to be a standalone product it supported https with a GUI wrapper around all of the certificate operations. This still exists in my open source version, making it easy to use self signed certificates for intra-device rest calls. But I've kept and expanded upon the multi-partition memory overlayFS approach and the installation of this system first asks you to install the sbts-base system, which installs the multi-partition memory overlayFS so that other's can use this as their own base systems.
- ikari_pl 3y ago
  I had a similar experience when I hacked a $5 Pentium 2 PC into a fanless (whether it liked it or not) and noiseless workstation. Replaced the HDD with a CF card. After a while, the system started stalling for 1-2s on disk writes, and that was a pain.
MPSimmons 3y ago
I am urging everyone who wants to do this to see if they can't first do what they need with a small board like an ESP32. Their energy usage is a small fraction, they cost ones of dollars, and they're sufficient for a whole lot. If you're of the Python persuasion, many boards support both MicroPython and CircuitPython. It's worth looking into for the cost savings on initial purchase and ongoing power draw.
- sofixa 3y ago
  ESP32 and related are pretty cool, but it's a whole different mindset; if something doesn't work, you can't just connect an HDMI/keyboard to debug live with all the regular utilities one might know and that come for free with any mainiline Linux.
  - MPSimmons 3y ago
    There are definitely limitations, but if it's a thing that you're going to leave untouched for years, it's worth looking at a device that will use 1/10th (or less) the power.
    - sofixa 3y ago
      Maybe, but if it is going to take you drastically longer to write the software because there's a smaller ecosystem / the stuff you need doesn't have a readymade library / you've never done C/MicroPython / there aren't the Linux tools that will help you debug or do a simple crontab to run your script on a schedule... The ROI might not be there even with the lower power consumption.
      - MPSimmons 3y ago
        That's true, and an important consideration. I'm not an ESP fanboy, but I really do appreciate it as an engineer for how simple and cheap the development boards are, especially considering things like ESPhome exist. It's kind of ridiculous how many projects can be built by just writing a yaml file and a little soldering - https://esphome.io/guides/diy.html https://esphome.io/guides/diy.html
tunnuz 3y ago
I wonder if Raspberry OS comes configured not to swap on the SD by default …
mvip 3y ago
We’ve been running thousands of Pis in production for about a decade now. We’re beginning to shift to x86. The price/performance isn’t what it once was for the Pi. I gave a talk about our experience recently at State of Open Con here (https://youtu.be/vX-qK9mxKZI https://youtu.be/vX-qK9mxKZI).
- spintin 3y ago
  You need to factor in usage, if you are idling a lot ARM > X86. And you need to look at longevity, there I suspect ARM will outlive X86 too. For modularity ARM > X86 too, because they are cheaper to have many small of. But for scalability (= business in the current economy) X86 > ARM. Also all graphs should be per watt, that 2 -> 4 is more performant is not news, that it is more performant per watt is! And if you did that you would see that Raspberry 5 is not getting as much per watt performance increase as it should. We have peaked permanently for the eternity of mankind. Let that sink in. Last but not least, the ONLY hope for any progress (openness not performance) at this point is the JH7110, but they are lagging behind in 3D support.
  - mvip 3y ago
    Our customer don’t really care about power usage much as they are all connected to a TV which draws order of magnitude more power. For some use cases power usage is key, but not for our use case.
  - AshamedCaptain 3y ago
    > if you are idling a lot ARM > X86. Don't take this for a given. RPis are notoriously bad at idle power consumption. The x86 replacement I bought for my home server ended up having about half the idle power consumption of the RPi it was replacing.
  - pi-rat 3y ago
    I think maybe you’re overestimating the idle power consumption of modernish x86 (especially the last 5-6 gens of intel)? I’ve a i5 9600T system that’s drawing less than a Pi at idle. And cost me the same to buy. https://docs.google.com/spreadsheets/u/1/d/1LHvT2fRp7I6Hf18LcSzsNnjp10VI-odvwZpQZKv_NCI/htmlview#gid=0 https://docs.google.com/spreadsheets/u/1/d/1LHvT2fRp7I6Hf18L...
askvictor 3y ago
I've been running a Pi 4 as a home server for a few years. When boot-from-USB became available, I moved to that, with a good-quality USB thumb drive for boot/root (I've had had SD card issues in the past) (and also speed, though I doubt it makes much practical difference). A couple of weeks ago I started getting intermittent disk errors. I thought it was the USB drive, so I cloned it (still worked well enough on my laptop) to another drive. Same thing happened. So everything points to the USB controller glitching out. Have gone back to an SD card, and everything seems fine.
- chopin 3y ago
  Have a look at your power supply. I had intermittent file system errors with cheap ones. I threw them all out if only for the fire hazard.
  - askvictor 3y ago
    Using the official pi 4 power supply.
cjdell 3y ago
I have an original Pi 2012 running buildroot. It's a perfect fit. Need just enough to run a Linux kernel and ser2net for doing RS485 stuff with solar inverters. I think the image size was around 100MB and no volatile filesystem whatsoever. Buildroot was surprisingly easy to use. Use a menuconfig to pick what you need and a burnable image for your SD card comes out the other side. Think I only spent an hour on the whole project.
archerx 3y ago
I have a cluster of 7 SBCs, 1 pi 3b and 6 Tinkerboards of various models. The Pi I got in 2016/17 and ran Gitlab without issue until 2022, something in the SD card got corrupt and linux was no longer able to boot, I was able to salvage all of the data and continued with a new SD card. The tinkerboards, which have various tasks as VPN, Nextcloud, staging servers using docker have never corrupted an SD card. I think the tinkerboards are better than Pi’s especially the ones that come with 16gb of onboard flash storage. However you don’t get all the niceness of PiOS but have to use TinkerOS (which was less than barebones when it came out) or Armbian which is nice but not built specifically for the tinkerboard. I have a few friends who complained about Pi’s corrupting SD cards and it also happened to my only long running Pi so there is something going on.
- LtWorf 3y ago
  I had an older rpi model without wifi where the network adapter got broken. Other than that, I've been running 3 rpi at home for home automation they have been working with no issue. But it turns out that they are a bit underpowered to be used as a minetest server. I never had a corrupted sd card on those. But I had an android nokia phone that corrupted like 3 sd cards before i gave up and stopped putting new ones.
spintin 3y ago
I have been running a Raspberry 2 cluster for 10 years. A few weeks back the first SD card to fail got so corrupted it failed to reboot! My key learning is use oversized cards, because then the bitcycling will wear slower! I'm going from 32GB to 256/512/1024! That said "High Endurance" cards are a scam, they fail way quicker than regular cards! All SD cards except SanDisk have latency problems. There is no competition. If you can get SLC SD cards use them for workload instances = no db or file storage.
exfil 3y ago
I run my RPi's 24/7 from initramfs. I can even remove card after boot.
- greggsy 3y ago
  Would you send them out to customers relying on them?
  - exfil 3y ago
    Yes. They use them like that.
TacticalCoder 3y ago
Anything long running shouldn't happen over that piece of crap that WiFi is. Data centers aren't build over WiFi. My network at home neither. I've been running RPi for years (including a VoIP server on a RPi 1!). The two tricks if you want a really long running Pi are: SD cards mounted read-only and ethernet. FWIW I ve got a Pi running the unbound DNS resolver and it just works. It is not an art. There s a reason businesses are scooping up millions of Pi: they just work. P.S: I ve got an army of NUCs too.
- LtWorf 3y ago
  The ethernet card of my 1st rpi broke :D
RobotToaster 3y ago
Anything long running should have ECC RAM, which the Pi doesn't have.
- anonym29 3y ago
  This is only a concern for workloads where the loss of contents of memory is a significant problem beyond the need to reboot, no?
blackfawn 3y ago
Their suggestion for log2ram does help for the most part but depending on what the pi is running, even that doesn't completely solve it. I've burnt out a number of microSD cards running a Pi-hole instance. I finally gave up and moved it to a tiny x86 with a SSD. Yet I've been feeding ADSB-Exchange and FlightAware from a Pi Zero for years and never had SD card problems. I really like Alpine Linux for the Pi, running in it's own read-only mode where changes must be committed to disk. But unfortunately, Pi-hole isn't compatible with Alpine (at least last time I checked)
not_the_fda 3y ago
All the SD problems go away if you get a RPi Compute Module instead. Well worth the additional cost.
- slow_typist 3y ago
  How do the problems go away? Is the lifespan of the built in eMMC longer compared to a high quality wear leveling sd-card? I am a layman, but isn’t the problem the flash which should be basically the same?
  - not_the_fda 3y ago
    An SD card is using contact pins to make the connection, which is susceptible to dirt and vibration. eMMC is soldered to the board so you no-longer need to worry about environmental and mechanical disturbances. The eMMC has a known supply chain and is guaranteed for 1 million write cycles that are then wear leveled. The SD card could be coming from anywhere, or a counterfeit with who knows how many write cycles and wear leveling. The CMs are built for industrial use and I've shipped a few products based on them, shipping thousands of units. They have had zero flash failures.
    - slow_typist 3y ago
      Thanks for elaborating, valuable advice.
    - megous 3y ago
      Not even NOR SLC flashes are regularly specified for 1 million erase cycles. And common MLC/TLC eMMCs certainly are not. What are you talking about? :)
hansonpeter 3y ago
[dead]
thehias 3y ago
And here is me, who is running a Pi1 in my cellar for 10 years straight, which logs all my temperature sensors over 433Mhz + triggers my door openers via physical relay over WIFI, without doing anything special. After some years I only connected it to a UPS, after the sdcard filesystem died after some power outages....
vagab0nd 3y ago
I use a USB connected SSD as boot. Been running Home Assistant and Pihole for years with zero issues. EDIT: Also, make sure the power supply is sufficient. I was using a cheap adapter and had random errors and reboots.
- chopin 3y ago
  Cheap adapters are a killer. I switched to a good 50W USB charger which powers a couple of Raspberries. Zero problems since then.
HankB99 3y ago
I use some Pis for various things in my house including Zeroes through CM4s and 4Bs. The Zeroes run Raspbian configured with the read-only filesystem option. I have found it necessary to uninstall `unattended-upgrades` because the overlayfs employed for read-only root caches disk writes in RAM and the update/upgrade process exhausts RAM. For the same reason I disable swap. It makes no sense to swap to RAM on a 512GB system. Upgrades are tedious since they require disabling overlayfs, rebooting, upgrading, rebooting, and enabling overlayfs. I wrote Ansible playbooks to perform these tasks. (https://github.com/HankB/Ansible/tree/main/Pi https://github.com/HankB/Ansible/tree/main/Pi) I have a Pi 4B performing as a file server and running Debian (not Raspbian) It boots from an SD card so that the entire HDDs can be used for a ZFS pool. To reduce wear and tear on the SD card I have mounted `/var` to a ZFS filesystem. I should probably use `tmpfs` for `/tmp`. I use a Pi CM4 to run HomeAssistant and that boots and runs from an NVME SSD where durability is less an issue.
lxgr 3y ago
> This is unlikely to do anything unless you’re hitting some unusual bug, but it’s worth noting that IPv6 has, in the past, led to all sorts of strange behaviors in different networking contexts. This makes me sad. I don't doubt that there are scenarios in which having IPv6 connectivity makes things worse, but these days, the opposite is more common, so I don't think "disable IPv6 just in case" is a good blanket recommendation to make anymore. "If disabling IPv6 fixes your issue consistently, consider disabling it" would achieve the same outcome, without potentially causing problems/inefficiencies down the road.
- dannyw 3y ago
  I agree with this. While I’ve had various issues with IPv6 a decade or so ago, nowadays I’ve never found disabling IPv6 to be a solution.
tgsovlerkhgsel 3y ago
I don't think I took many special precautions and a Pi I had running as a VPN server survived for about 10 years, aside from a power supply failure or two.
tzs 3y ago
A few people have mentioned achieving long uptime. What is often overlooked is that it is possible for uptime to be too long. It is quite possible for updates to not break a running system but make it so that it will break on the next reboot. E.g., a dynamic library gets updated in a way that breaks a server process. It doesn't affect the running server because it still has the old library loaded. Next time you boot your server process doesn't start. These kind of problems can be annoying to deal with, especially when your system has an uptime of years and for all you know whatever change broke it could have been in any one of dozens of updates you've applied over that time.
- pmontra 3y ago
  How do you deal with that? Probably one should have a canary system that is rebooted every day. In a home setup we usually don't hand either the spare machine or the spare time to deal with it, or both.
  - tzs 3y ago
    The easiest way is to reboot after updates. Most updates are not urgent so unless you are having some problem that you hope the update will address wait until a time when it is OK to reboot. Then install the update and reboot. If there is an urgent update that you need to apply at a time when rebooting would be an issue apply the update and leave yourself a reminder that the system is running a configure that you don't know is bootable. When a reasonable time for a reboot comes around take it.
  - progman32 3y ago
    Me personally, I always reboot after every update. I'd rather deal with the break now while I'm working on it, rather than at some random point. Then I'll know exactly what changed. I take it a step further. I usually reboot before updates, too. Make sure I won't chase updates as an issue if something else broke.
hn8305823 3y ago
> This feels like a hack, but based on hours of reading online discussions, most people seem to settle on a script that periodically checks whether the WiFi connection is good, and restarts the WiFi interface or the whole Pi if it’s not. It's not a hack, it's best practice! Just like important servers in a data center should have some kind of out of band connectivity (IPMI, remote controllable RPDU outlets, etc), Important servers in remote difficult to reach locations should have some kind of watchdog script. The script should of course be tuned to the specific use case, considering the impact of a reboot vs downtime until reboot. At the very least it could log adverse events for later investigation. A simple bash watchdog script was the very first thing I did when I deployed a remote RPI. Not just for wi-fi issues but for any of the dozens of things that could break and be fixed with a reboot.
- vb6sp6 3y ago
  [dead]
- slicktux 3y ago
  Totally agree! Watchdog timers are essential for microcontroller and even computers running software “forever”. Things do happen that even perfect code and design can’t prevent and a watchdog timer will break out of an infinite loop and reset. Things such as cosmic rays flipping a bit or even brownouts…in raspberry pi you also have to worry about SD card corruption… Edit: raspberry pi’s have built-in hardware watchdog timers I believe. I know Arduino's do!
- bravetraveler 3y ago
  Nowadays this watchdog is init/PID1, on most distributions - systemd. If init can't be relied on to manage services, what guarantees do you have for the system to provide them? Sure, one could reinvent this in scripts, but we've moved past this. I mention systemd a lot, but that's not to cast favor - there are alternatives. Most services don't make appropriate use of the environment they exist in. I assume they expect some site customization, ie: declaring your web server needs these mounts. A commonly overlooked directive is 'PartOf='. You can tie restarts of one service/resource to another. Heck, more simply, I think NetworkManager offers a way to customize the wifi/portal checking. You may not have to go completely heavy-handed
masto 3y ago
Step 1 is to question why you're using a Raspberry Pi. It's almost never the correct answer. If there's a really really good reason, step 2 is to get rid of the SD card. Personally, none of mine even have an SD card inserted (ok, one does). I use network boot/NFS for everything. Some people attach other kinds of SSDs. The one I lied about is a reverse telnet server that has been quietly doing its thing for 4 years without a hitch, apart from the time I had to replace the (PiJuice) backup battery because it was looking a little swole. But I should take the time to at least have a backup of the card ready to swap out when it fails.
- bricee98 3y ago
  I’m trying to make a smart speaker where you push a button to talk and when you release it sends the audio to a server for transcription and response. It also has to do some basic logic with LEDs. I also want it to be always on and available as long as it’s plugged in. Do you have any advice on what might be a better alternative to a Pi? This is basically my first foray into hardware so I’m trying to learn as much as possible!
- ianburrell 3y ago
  There are three uses for Pi. One is cheap computer, but Pi is never good at that and miniPCs have gotten cheap. Two is IoT, ESP32 is good choice for simple things and Pi Zero for bigger things. Three is for small servers and dedicated computers. Pi is perfect for that, it is cheaper than miniPCs, is smaller, and uses less power. Lots of people in this post have mentioned uses like that. I want to make outdoors ADS-B receiver, and Pi will fit in enclosure and be powered by PoE. I want to make GPS time server, and Pi has PPS input on header. I want to make portable ham radio box, and Pi can be powered by 12V DC.
cjfd 3y ago
My experience with long-running raspberry pi's is that there are USB problems. One sometimes loses the connection to these devices.
jokoon 3y ago
I don't understand what's hard about adding like 256MB of flash soldered to that thing, I'd pay $5 more if it did. Everytime there's a RPI I cry about it.
- justin66 3y ago
  Judging from a sampling of your comment history, every time someone points out that there's a Raspberry Pi compute module option that comes with flash memory.
greenie_beans 3y ago
eventually you'll think about power outages. i had a pihat battery nearly explode on me. beware!
anonym29 3y ago
Do people really have such a neurotic aversion to the sight of cables that they'd rather struggle with workarounds to an always-inferior wifi than just using an ethernet cable? It seems unhealthy to get that stressed out over the sight of a 3cm wide cable. I've had an RPi4 running continually for close to a year and never had a single network connectivity issue between the RPi4 and my router, connected via a 30m cat5 cable in my 75m² condo.
haunter 3y ago
I started buying Lenovo mini PCs instead, 18cm x 18cm x 3cm so it's still really small. And you can get them dirt cheap nowadays, has proper casing and cooling etc https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCentre_M720_Tiny/ThinkCentre_M720_Tiny_Spec.pdf https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCe... I have one right next to me, i5-8500T, 32GB RAM, 2x SSDs and currently 5W at idle with powertop auto-tune https://wiki.archlinux.org/title/powertop https://wiki.archlinux.org/title/powertop
- thedougd 3y ago
  Same. Bought a Dell refurb and I’ve had so many fewer hassles. The SD card was perhaps the biggest source of issues and after factoring the cost of outfitting a RPi with something like an SSD, it just wasn’t cost effective anymore.
  - mbirth 3y ago
    This I never understood. I have 3 different RPis running since they were available to buy. A 2, a 3 and a 4. All with the OS on microSD and no optimisation like writing log files to tmpfs or similar. It’s plain Ubuntu Server for ARM. And in all these years, I had maybe one microSD going read-only on me. Cloned that to a fresh one, did an fsck and the Pi was up and running again. That’s it. No other issues despite various sudden power losses etc. I don’t understand where these rumours about bad reliability of SD cards in Pis come from.
    - thedougd 3y ago
      Maybe you're lucky or buying the right SD cards? I've always used noatime for filesystem, journald set to ram logging, etc. Usually everything seems fine until a reboot triggers a fsck. Most were ext 3/4 filesystems.
    - ecef9-8c0f-4374 3y ago
      Pi 1 a/b were really fragile but since pi 2 this problem is mostly solved. Still if I need to write a lot I would use a external drive. We have >300 pies running 24/7 on 52 sites all over the world. Even though we use no name brand microSDs we had only 2 failures because of a broken SD card in 4 years. Our partner company was using our hardware as a basis for something. After 1 month they asked us what SD cards brand we were using because they had a high failure rate. Our secret. We use a self made distro with minimal writes, they used default raspian and were writing permanently to the SD card.
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
psanford 3y ago
Switching to gokrazy[0] was the best thing I did for my Raspberry Pi uptimes. I think a lot of that is because it defaults to using read-only partitions so the common issue of SD cards falling over when you run apt upgrade no longer happens. But I also think that gokrazy's simplicity and design helps it be just a solid, reliable foundation to build on top of. [0]: https://gokrazy.org/ https://gokrazy.org/
- whitehexagon 3y ago
  Looks interesting! but not clear to me who is behind the project? do you know if it is a commercial sponsorship, Golang team / Google, or pure community based open source ?
  - psanford 3y ago
    There's no company behind it. Its a personal project by Michael Stapelberg.
- NortySpock 3y ago
  I used dietpi [1] for similar reasons: a slim version of Debian, and with the defaults set to push all the logging into ram to minimize writes. Dietpi has opinionated defaults, for sure, but it's easy to choose something else (e.g. Dropbear is the default ssh server, but bumping to OpenSSH is a matter of changing a setting in the handy config tool). I've been running an RPi3 on dietpi on an SD card as my secondary PiHole instance on it for at least a year with no issues. [1] https://dietpi.com/ https://dietpi.com/
nodesocket 3y ago
I can appreciate the effort and time put into this, but seems completely overkill. I’ve been running 4x Pi 4B’s in a Kubernetes cluster for over 3 years 24/7/365. Only thing I did is disable swap (which you should do for Kubernetes anyway) and used SanDisk Extreme 128GB cards[1]. [1] https://a.co/d/ipehodH https://a.co/d/ipehodH
deleted 3y ago
[deleted]
cattlepi 3y ago
Going to plug my own side project from the past here: https://cattlepi.com/ https://cattlepi.com/ https://github.com/cattlepi/cattlepi/blob/main/README.md https://github.com/cattlepi/cattlepi/blob/main/README.md Been running pis (mostly 3b+) for eons with this solution and at this point i can say it's bulletproof. The key is to minimize the sdcard wear and tear (it uses an overlay filesystem with squashfs as base and tmpfs as write top layer) and to keep zero stare on the device. You can build the image starting from normal raspbian. You can also update it over the network. As far as usage in the wild the largest "deployment" (i know about) is at around 1000 pis.
Zopieux 3y ago
Not a single mention of netbooting the Pi instead of relying on crappy SD cards which wear faster than I can type this.
vikmals 3y ago
> Your SD card can wear out or completely fill up Why is the author not considering using an SSD instead of an SD-Card here?
WaitWaitWha 3y ago
It is the the time for mini PCs as many have noted it. I am getting one for one of my offices, instead of building or buying a desktop. I used to build my machines and loved it. Some are still chugging along after over 10 years. They were worthwhile before the mini PCs reaching comparable capabilities at lower cost. One thing I still cannot seem to find is a good site that compares various vendors in similar fashion as "PC builder" sites do of build components. Any suggestions?
1vuio0pswjnm7 3y ago
"Keeping a Raspberry Pi online and working with zero intervention for weeks, months, or years is somewhat of an art form." I just boot NetBSD kernel with embedded filesystem, e.g., INSTALL kernel or custom kernel. SDCard can be removed immediately after boot. Optionally chroot to attached storage. This runs for weeks, months or years. Have not experienced any of the issues cited by the blog author. Only issue I find is with the power connector when using a case; the connection can be brittle, e.g., if using a replacement cable. Perhaps this has improved on more recent Pis. (But I could say the same about most computers. The cables and connectors are usually fragile. It's always cheap stuff.) If power is interrupted because of movement, then the Pi reboots automatically.
ChuckMcM 3y ago
The longest running RPi I have has run continuously for over 5 years. The big secret was not to use the SD card at all[1], I mount all file systems over the network to a NAS device (TruNAS from iX systems). It has a "UPS" in the form of a USB battery pack that is both charging from main power and powering the Pi. When power goes out the battery pack takes over, it has about a day of "hold up" time depending on Pi power usage. It is 'hard wired' to the local network (it doesn't use WiFi). I got there from having SD cards (nearly[2]) always be the failure point. Everything else has been pretty reliable when used within tolerances. [1] It does "boot" from the SD card but that acts kind of like a third stage bootloader which loads and boots the "real" OS (FreeBSD) from the NAS device. [2] I have had one fairly spectacular looking "melt down" of a no-name USB power supply wallwart PSU which, to appears to have also put something like 12V directly across the USB power pins (my best guess at what the secondary winding of the xformer in the wall wart was putting out on the 'low' side)
geon 3y ago
When I worked with that, we had the SD card entirely read-only, and a usb stick mounted for writing. I think the rpi wouold detect a broken file system on boot and reformat the usb stick. It also made it easy to pull the usb stick and format it in any windows pc to reset it.
husam212 3y ago
I use OpenWrt whenever it's possible, to avoid SD card wear completely.
pythonsnake 3y ago
Not sure about the SD cards he recommends. Use Swissbit SD cards. But they will cost you. Mouser is your friend.
demondemidi 3y ago
> Keeping a Raspberry Pi online and working with zero intervention for weeks, months, or years is somewhat of an art form. What are you talking about? It is literally zero effort. Just set up crontab reboots in case of power outage. That's it. I've had a pi running as a BLE gateway / security cam for over 4 years with zero intervention.
Animats 3y ago
Does that have a hardware stall timer?
jimmyswimmy 3y ago
Around early 2019 I set up a raspberry pi 3 running Raspbian. I made the /var/log partition a ramdisk. Haven't touched it since. It goes down for power outages but probably has been out for a couple minutes total over five years (aside from power outages). Most of its job is to translate analog audio to a USB speaker system. Whole house audio for about $150. Anyway, I never touch it, it just works, all the time.
jnaina 3y ago
I have a Raspberry PI 1 model B running almost non-stop for over 8 years (except during power outages) serving as my front gate controller. Custom python code uses the GPIO pins to trigger an RF remote to open or close the gate, and does not write to SD for logs/etc (to prevent SD wear) I use a USB network card, and a high-quality SD card. Other than that, no other special configs (except for another spare SD card with a full system image clone). Rock solid performance and uptime over 8 years.
rldjbpin 3y ago
having used an original model b (rev 2) since 2013 with running 24/7 for about 80% of the time, i feel that a lot of this depends on how much you want to push your pi. for context, my workload was bursty in nature and nowadays it is mostly idle. but i never had any issues with the SD card, and i only upgraded to a new one for getting more capacity all this time (only twice). keeping the dust out and managing the temperatures would go most of the way. i have served files directly from the sd card but it is always better to mount an external drive for this, while providing enough power to the board to power usb devices. limiting debug logs for stable applications can also help avoid write cycles, but using sd card on a pi has been a similar workload to using one in older smartphones for storing media.

510 points by ilikepi 3y ago

As far a I know (and my knowledge inly goes to version 4) the RasPi does not support different power states or sleep modes. It would be interesting how it compares in power consumption to other solutions.

_cenw 3y ago

Alpine in diskless mode and an ethernet cable solve 2/3 of that.

agilob 3y ago

Half of the post is about SD cards, wear, data loss and reliability. Just use SSD?

teekert 3y ago
I was expecting that to be tip #1 indeed. And it’s why I went to a Nuc eventually. Choose a nice ssd and it will really run for years and years.
- yjftsjthsd-h 3y ago
  Nucs also tend to win on performance, software compatibility/upkeep (yay commodity x86), price per performance, and often price outright (pis aren't really $35 if you get a newer/better model and once you buy storage/power/case). AFAICT pis only consistently win on power consumption, and that only just (x86 can get well under 10W, you just have to get the right model)
  - arlecks 3y ago
    Do you have recommendations for a NUC to replace a raspberry pi that draws around 10W when idle?
    - johnchristopher 3y ago
      And possibly one as silent as a fanless pi4 ?
    - enioarda 3y ago
      Have a look at the Beelink SER5, idle around 5W and its quite affordable https://www.notebookcheck.net/Ryzen-5-5560U-performance-debut-Beelink-SER5-mini-PC-review.762934.0.html https://www.notebookcheck.net/Ryzen-5-5560U-performance-debu...
    - dannyw 3y ago
      The Intel N95 and N100 NUCs are amazingly efficient.
- CTDOCodebases 3y ago
  Thin clients are also an option. Similar power consumption but power and case/cooling/storage is included. Depending on what you choose you get eMMC or SATA or M.2 SATA or M.2 NVMe for storage. For anyone that is interested: https://www.parkytowers.me.uk/thin/ https://www.parkytowers.me.uk/thin/
srvaroa 3y ago
Maybe professional bias but I prefer to follow a cattle vs pets approach here and treat them as disposable, meaning: let the sd card break and optimize for quick replacement.
- alpaca128 3y ago
  Current Pi 4s & 5s support booting from USB out of the box with no configuration (took them long enough), so I don't think it's worth the downtime and wasted SD cards.
- imtringued 3y ago
  NVMe SSDs cost the same as SD cards nowadays. You can treat the SSDs as disposable if you want, but you are going to quickly realize that this mindset isn't exactly actionable, because there is nothing that needs disposing of.
imtringued 3y ago
Before I opened the article I was thinking "Don't use an SD card. Don't use an SD card. Don't use an SD card." People don't understand that 90% of the SD card problems are power related (only relevant if you don't use the official power supply) and 10% are simply because of poor SD card quality. People haven't gotten the message that a charger has lower QC standards since an interruption of the charging process does not shut the device down. A bug that leads to a few milliseconds of power loss will pass QC, but also corrupt your SD card.
- dboreham 3y ago
  Penny drops. Yes this jives much better with the observed data -- that Rpi break SD cards orders of magnitude more than anything else that uses SD cards no matter how crappy said cards are. Problem could have been solved by adding a decent sized capacitor on the 5v supply rail.
  - justin66 3y ago
    > that Rpi break SD cards orders of magnitude more than anything else that uses SD cards I think if you take into account how often they're powered on you'd find cameras destroy flash memory cards comparably to Raspberry Pis. > Problem could have been solved by adding a decent sized capacitor on the 5v supply rail. Or you could just buy the Raspberry Pi branded power supply.
- pmontra 3y ago
  Are SSDs less prone to corruption than SD cards when power drops for those milliseconds?
TaylorAlexander 3y ago
You can also use a Raspberry Pi compute module in an appropriate carrier. I use the CM4 for work and consequently it’s all I use at home too. Lovely versatile device that fits in whatever carrier you want (including one that mocks a regular pi) and it has industrial grade flash.
- youngtaff 3y ago
  Do you have any recommendations for a carrier?
  - johnchristopher 3y ago
    What's a carrier in that context ?
    - bemusedthrow75 3y ago
      An IO board that adds the IO ports that the e.g. CM4 lacks. There's a decent range of them from different suppliers, but Raspberry Pi have one themselves called "Raspberry Pi Compute Module 4 IO Board".
      - johnchristopher 3y ago
        Thanks !
  - TaylorAlexander 3y ago
    I use this minimal one. Check out what Waveshare has to offer too. https://sourcekit.cc/#/ https://sourcekit.cc/#/
    - youngtaff 3y ago
      Thanks… my ideal would be a similar sized carrier with PoE and M2 support Think I really want something like the Uptime Labs Blade https://pipci.jeffgeerling.com/boards_cm https://pipci.jeffgeerling.com/boards_cm

demomode 3y ago

Also: consider using overlayfs to make root fs read-only

CTDOCodebases 3y ago
They mentioned making the root fs read-only which (if they are using the config tools) does use overlayfs: https://raspberrypi.stackexchange.com/questions/124628/raspbian-enable-disable-overlayfs-from-terminal https://raspberrypi.stackexchange.com/questions/124628/raspb...
devishankar 3y ago
[flagged]

yjftsjthsd-h 3y ago

I'm shocked the SD card bit isn't first, and more surprised that the post doesn't suggest USB boot (I have one pi that's been on ~24/7 for years now, and I attribute its lack of problems to 1. using Alpine configured to barely touch disk, and 2. not having an SD card to corrupt - I don't know why USB would be more reliable, but anecdotally it is)

Tor3 3y ago
I've had a bunch of Pi cards, running on SD without problems. But a single one suddenly developed a super-hot SD card, this was a brand new Pi which I was just setting up. Got the card out, and that one and the next Pi got a USB SSD and are now using those. That was a bit scary. But as mentioned I've also been running Pi with micro SD as Cups servers for years, with no problems at all.
- Aerbil313 3y ago
  Would there be any reliability problems with using USB flash drives instead of USB SSDs?
  - Tor3 3y ago
    I don't really know. I assume they would be slower than SSDs though, but I have never measured USB flash. The SSBs get some 300MB/sec on my Pi 4 boards.
  - NortySpock 3y ago
    Sample size of 1, but I ran an RPi4 just fine on a USB thumb drive for about a year. Still upgraded to an SSD later because I wanted even more storage space (and SSD seemed to have better random IOPS than a thumb-drive), but I'd say go for it.
breton 3y ago
I agree with this completely. All my rpi failures were because of SD cards. I have 2 rpis, both boot and run from usb, both for several years now.

goodburb 3y ago

I'm still using a Sandisk 8GB microSD from 2008 running 24/7: smartphone expansion 6 years -> Orange Pi as a router for 3 years, Pi 4 running Frigate for 3 years. I'm guessing it's SLC/MLC. Had a Transcend 32GB in 2016 die after a year. The biggest issues with set-and-forget setups is software upgrade for security or other reasons, jumping major versions ends up breaking things. Compared to cloud which is (usually) regularly updated.

adamm255 3y ago
All my SD cards are Sandisk in all my Pis. (2x B+ and a 4, and a Zero). Never a single issue in over 10 years of running at least 1 Pi 24/7. Only buy Sandisk I guess!

niederman 3y ago

Admittedly, I only have one long-running Raspberry Pi, but it's currently sitting at a few months uptime. And that was an intentional reboot. I've never had to take any measures like these in the four or so years I've had it up.

guenthert 3y ago

No mentioning of flashybrid? I'd thought that's the obvious solution to SD wear (or rather the danger of SD corruption on sudden power loss).

guenthert 3y ago
Too late to edit, but I meant to refer to the Debian package of that name, not an actual flash/spinning rust hybrid storage device.

vegabook 3y ago

a) Cable ethernet b) SSD (via USB3.0 adapter on my RPI4) c) Ubuntu Server LTS 22.04. d) cheap UPS. Mine runs Yggdrasil network, HAproxy, Caddy server, a couple of webservers in containers, and a TMUX instance that I log into almost daily to write code (slow computer reveals bad code much better). Since I put it (and my router) on the UPS, in the last 2 years it has literally never gone offline other than a couple of times I rebooted it for firmware upgrades.

MOARDONGZPLZ 3y ago
This is mostly what I do for “critical” Pi things. The cable Ethernet and the SSD are major. Do you have any recommendations for a cheap UPS? What are your considerations here?
- vegabook 3y ago
  I went with a CyberPower BR1200 which was about 150 pounds (circa 200 USD) but that's overkill because it will keep my pi+router going for several hours, whereas power-downs typically last less than 15 minutes max where I live and I haven't seen > 1 hour in 10 years. It's usually "human error" related someone plugs the vacuum in or some appliance trips a circuit. Keeping the router on the UPS (and fiber box in my case) is important though because then everything is completely uninterrupted from a serving perspective. I also give the standard RPI power adapter enough headroom by not overclocking the pi, because in the past this has caused problems when SSD attached with lots of writing, and "big" (for a pi) compute load at the same time. Since turning off the overclocking, zero problems. Probably an RPI5-class power supply would be even better. No clue if this matters but on a jetson nano (which was notoriously power-spikey), crucial (micron) or samsung SSDs tended to be better than budget alternatives.

akira2501 3y ago

I got one of those super low profile USB thumb drives and then set the Pi to boot from that. It ran automation here in my house for 2.5 years without a blip.

guenthert 3y ago

Also no mentioning of the power connector? I have too little experience with USB-C, but the micro USB connector used on early Raspberry Pi's is just asking for trouble. That might be (barely) good enough for charging, but a computing device w/o battery-backup won't take lightly the power interruptions when jiggling the cable a little. I finally got around to replace it with an old-fashioned (time tested!) barrel connector. Easy way to improve the robustness significantly.

bemusedthrow75 3y ago
As with my DSLRs/mirrorless cameras (particularly the ones that use the wretched USB mini) when tethering I have ended up with a right-angle adapter cable secured with a cable tie (or the super-strong Tethertools Jerkstopper), and a USB micro cord into that. USB micro is designed to snap at the connector, not the board. And the connectors will indeed snap.

FerretFred 3y ago

I've been running a bunch of Pi's for years now, and the biggest problem I've had is the Pi itself dying: 24/7 usage is hard on a small device. I've also found that stable power is essential, and to that end I've always used 5v 3a branded power cubes, plugged into a pure sine wave UPS. Choice of micro-SDHC cards is important and I ended up getting ATP industrial cards (https://www.atpinc.com/products/industrial-sd-cards https://www.atpinc.com/products/industrial-sd-cards) - expensive but really long-lived. Finally, using RPi-clone (https://github.com/billw2/rpi-clone https://github.com/billw2/rpi-clone) on a regular basis has been a life-saver. I clone to Sandisk Extreme micro-SDHCs and can recover from an outage in minutes.

vietvu 3y ago
And how frequent you write to SD card too. My Pi 3B+ have been running for 3 years. Haven't considered to upgrade since my need is small.

rcarmo 3y ago

I did exactly zero of any of those things and have had some Pis run for multiple years without any issues until being replaces by a newer model (my HomeKit/Zigbee gateway and data logger is now a Pi 4). I guess it all boils down to good SD cards and stable power supplies.

Tor3 3y ago
Same here - I've been running a couple of Pi3 as Cups servers for years of uptime (the only time that uptime gets reset is when there's a power outage - and that's very rare indeed). Did nothing more than install Raspbian on a micro SD card, set up Cups, connect USB to printer (for one of them - the other manages a networked printer). And left them alone after that.
dgellow 3y ago
My Raspberry Pi 2 ran for a while, used as a pihole, a VLC-hacked media center, and a weather stations for simple sensors, until it didn’t :( In ~2022 I started to get random errors, then found out the SD card was failing. I never took the time to fix it, it had lots of little things locally I don’t feel like doing again.
- domh 3y ago
  It may seem like overkill, but I learnt how to use ansible to manage the software on my pi. I've actually just upgraded my rpi4 to use an ssd so am choosing to start from scratch. I'm hoping my ansible playbooks are written well.
  - dgellow 3y ago
    Haha, yeah, that’s a good point. I may still have some ansible playbook from a while ago but I’m pretty sure it wouldn’t be up to date. I guess it would be nice to have a system like CoreOS was, where you just provide a setup script, systemd unit files, and at runtime most of the file system is expected to be read-only. That way you’re confident you keep all your setup in a git repo and on reboot the whole thing is reset (outside of data stores). It’s just so tempting to quickly ssh into the machine to hack something around, then you forget about it b cause it ~works. But a rpi4 can run containers, so that may also be an alternative.

hcfman 3y ago

My sound localizing Raspberry Pi installs a resilient base system as part of its install. https://github.com/hcfman/sbts-aru https://github.com/hcfman/sbts-aru https://hackaday.com/2023/12/30/localizing-fireworks-launches-with-a-raspberry-pi/ https://hackaday.com/2023/12/30/localizing-fireworks-launche... With one command it for all Pi’s for both Raspbian and bookworm it: * Shrinks the file system (Gee, how does it do that with just one disk ? ;-) ) * Creates new partitions * Installs a memory overlayFS * Installs and configures the system as an audio recorder with micro second time accuracy * updates /etc/rc to do a forced repair of the data and config portions, in case they were damaged. This avoids system hangs waiting for human interaction with fsck For the partitioning scheme it creates a swap partition, not as a wow but as an enabler if you really need it to install some large software. It creates a small config partition. The idea here is that you keep it read only and remount it read write if you need to change config then remount it read only again. And finally a data partition, which in this projects case is where the audio files are written. I maintain a version of an overlayFS boot for the Pi but it needs revisiting for bookworm. The easiest way to use do this is to install the sbts-aru and then just don’t use it. Then everything is done for you in one command. And that version works for all Pi’s. I also do this for the Jetson SBCs. But I need to revisit this for the Orin series. I have it working here for myself and friends but need to update the installer. Note, due to kernel behavior changes with Orin the older Pi like overlayFS code will not work. But I solved this and will release it when I release the Orin release of sbts-install soon. I’ve been using memory overlayFS like installs for years for long running Pi systems.

teleforce 3y ago

For the life of me I don't really get it why Raspberry Pi Foundation does not include onboard eMMC or SSD storage in their non-compute modules products. Yes with the new PCIe expansion in the latest RPi 5 you can have external SSD for example, but if you decided to use it for other purposes as well like extra Ethernet port expansion then you cannot use it for booting anymore.

bemusedthrow75 3y ago
> For the life of me I don't really get it why Raspberry Pi Foundation does not include onboard eMMC or SSD storage in their non-compute modules products. Cost. It's always cost. If the eMMC or SSD storage is not enough to hold a general purpose OS for all users, then only some users get the value. And if it is big enough, it's put the cost of the machine up above the point at which they feel happiest, when an SD card is perfectly fine for the majority of their target users. Eben Upton is regularly on record talking about how the cost-per-component/users-per-component tradeoff will lead them to avoid adding a component, and has motivated removing some (composite video for example)
justin66 3y ago
It's obviously about cost. It's curious that there are people who need this, are aware of the compute module, and still complain about it. Is the the availability of the compute module that's a problem?

Alifatisk 3y ago

Haven’t followed any of these tips, yet I have barely stumbled upon any issued with my long running pi

blacklion 3y ago

First advice is to enable journaling mode on FS. First advice must be to mount FS in read only mode, mount /var in memory and forward al logs to one nide, which may be not RPi but something with proper UPS and nut running. Power loss becomes absolutely bening if your FSes RO or temporary. It is overkill if you have one RPi but author claims that he uses multiple RPis all around a house. Also, good idea to have A/B system partitions and upgrade system with full partition rewrite and changing active one. Thus way your system will have one good system partition in any case, even if new version has fatal bugs, and recovery become trivial. I'm using several small/single board PCs im different roles in such way for 20+ years with great success.

Avamander 3y ago
In addition to /var, tmpfs should also be used for /tmp and similar. That should lengthen the SD card's lifetime immensely.
- diggan 3y ago
  Isn't this (tmpfs for /tmp) the default setup for most OSes, and surely raspberry's homegrown OS too?
  - tyingq 3y ago
    It seems it is not the default for Raspbian.
    - diggan 3y ago
      TIL! One would think if any distribution had it by default, it would have been Raspbian.
      - tyingq 3y ago
        Maybe left off since some of the models have very little memory to spare for a ramdisk?

tempaccount1234 3y ago

I have a few long time running Pis - and have been keeping them up for a decade now. No SD Card corruption ever and i got close to 1000 days uptime on one. The biggest problem is loss of wifi, after a few months one will lose wifi, but keep working - it’s constantly recording data so a reboot is not a good idea. I’d prefer a solution where I could just reset the wifi, but all attempts to script that reliably so far failed.

nottorp 3y ago
I've got a few in factories (doing non critical stuff) at work. They auto reboot regularly because otherwise they'll lose whatever's on usb (reading those usb metering devices being the whole point of the PIs).

Namidairo 3y ago

I'd consider enabling the hardware watchdog as well. While one could argue that you should figure out the source of your device freezing in the first place; Nothing is better than having to ask someone to power cycle your Raspberry Pi while you're away.

drpixie 3y ago
Yes. And I monitor my network link and automatically power-cycle the modem, if the worst happens. (Which is rare, but the network link has been the source of most of the few problems I've had.) And use a wired network connection for anything important!

Temetra 3y ago

I've been plagued with the wifi problem since changing routers. Devices on the local network will randomly lose the ability to connect to it, but everything else is fine.

forinti 3y ago
I added a cron job to one Pi that checks if wifi is up and tries to restart it if not. I also had an LG monitor that had so much feedback it would disable the WiFi interface completely. So I would check your monitor, if the pi is connected to one.

borlox 3y ago

Just read the data sheet of the Sandisk Max Endurance and, oh boy, what a fsck’ing marketing bs language. They state the endurance on thousands of hours of FHD video, but what assumptions do they make in bitrate etc? Can‘t they state total TB written or drive writes per day or something sensible?

userbinator 3y ago
Flash memory manufacturers are notoriously secretive about the actual endurance of their products, likely because it's now embarrassingly low. Specifying endurance in "thousands of hours of FHD video" implies large sequential writes, or in other words the best-case for write amplification.
- dannyw 3y ago
  They also substitute parts all the time. Even the “best” brands have been known to do this. From different controllers to swapping TLC for QLC flash.
Shakahs 3y ago
Better manufacturers do give you a TBW endurance rating, and document what type of flash is being used (SLC, MLC, etc.). I used one of these Micron cards in my RPi4, it has high write endurance and also an A2 performance rating so it can support the IOPS needs of a boot drive. https://www.mouser.com/new/micron-technology/micron-i400-microsd-cards/ https://www.mouser.com/new/micron-technology/micron-i400-mic...

yokoprime 3y ago

I’ve got 2 rPI’s thats been running for 4+ years each. Only thing I do is update them once in a while. Both have workloads running 24/7. no issues. I have however experienced issues in the past but those were due to a faulty power adapter not keeping voltage within spec.

userbinator 3y ago

It's worth noting that SD card firmware is usually optimised for the FAT32 filesystem, which has a very predictable access (specifically write) pattern, and using filesystems that have a more "free-form" layout can lead to lower performance and higher write amplification: https://lwn.net/Articles/428584/ https://lwn.net/Articles/428584/

goodburb 3y ago
Good note, I wonder if wear leveling depends on that since it's local wear leveling compared to SSD global. SDXC probably requires exFAT and free space bitmap is considered.

sdflhasjd 3y ago

This covers the readonly filesystem, but doesn't cover the write protect flag that you can set on the microSD card itself[1]. The flag will configure the card's controller to drop any writes, and is thought to resist the corruption issues that can still occur even when the filesystem is readonly. Also, creating a readonly root out of an existing disto is a bit of a pain, my preference is to use a distro (like TinyCore) that's already a readonly root. https://github.com/BertoldVdb/sdtool https://github.com/BertoldVdb/sdtool

zh3 3y ago

As someone using many, many Pi's at home and many times that at work, the preferred approach is to boot them all diskless (and for those that actually need an SD card, boot them off a read-only SD card and get everything else off the home server). This is so much easier than having lots of different SD cards/Pi versions etc, and makes them trivial to replace in the event of failure.

isoprophlex 3y ago
I have several of them, sounds ideal. can you perhaps link me to a good/useful writeup on how to accomplish this?
- zh3 3y ago
  Look up diskless booting, it's a very general thing on linux (i.e. not Pi-specific, although there are plenty of Pi-specific tutorials on the 'net. As a minimal first step, install an NFS server (which can be X86, Pi, other) on your LAN and make sure you can mount it from the pi ("mount $SERVER:/some/dir /mnt/tmp"). Then copy the contents of a Pi SD card to the server, make it exportable (see '/etc/exports') then edit '/etc/fstab' on the Pi to mount the (now remote) copy of the SD card instead of the usual root. That should get you started - beyond that, with some of the Pi's you don't even need to have an SD card installed (however you'll then need to set up things like DHCP and TFTP on your server).
  - isoprophlex 3y ago
    Thanks for sharing!
  - nsteel 3y ago
    There are also some pi-specific bootloader config options that can make life easier, they are documented at https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#network-booting https://www.raspberrypi.com/documentation/computers/raspberr... and from https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#TFTP_FILE_TIMEOUT https://www.raspberrypi.com/documentation/computers/raspberr... onwards There's also a tutorial at https://www.raspberrypi.com/documentation/computers/remote-access.html#network-boot-your-raspberry-pi https://www.raspberrypi.com/documentation/computers/remote-a...

LeoPanthera 3y ago

All my home Pis network boot, so there is no card to fail. You can also change what OS they boot into by just renaming a symlink on the server and rebooting them. Very convenient.

jon-wood 3y ago
I'm curious what you're using all these Pis for when you've got a server on the network to provide network boot. Can you not just use the device that's providing network boot to provide whatever you're running on the Pis?
- LeoPanthera 3y ago
  No, because the server is in the wrong place. Some Pis are connected to televisions and play videos, or emulated games. One is connected to my hi-fi. One has a GPS receiver on it for NTP. One is a photo frame. Basically, the Pis exist to interface with the real world.

csydas 3y ago

i very much so enjoyed these articles even if i don’t use raspberry pi anymore. it has a very much so research notes cleaned up for sharing feel and reminds me of some of my teams internal articles, particularly on research the author wasn’t sure on but wanted to put out there regardless. it was very easy to get their decisions and the limit of their research and knowledge so i had pretty clear idea what i need to still check on my own and what i might adopt if i were to use rpi devices. i don’t really get what to use a rpi for but i guess not important, it was just a nice series of articles

rcMgD2BwE72F 3y ago
I can't do without my https://en.wikipedia.org/wiki/LibreELEC https://en.wikipedia.org/wiki/LibreELEC RPI.

lqet 3y ago

I have 2 Pis running basically non-stop (2-3 power outages) with the same SD card since 2017 (DNS/print server and Kodi, media is on external NFS). The only thing I did was to disable all logs. Never had a single problem. They both have SanDisk 2 GB cards in them. I vaguely remember naively thinking along the lines of "less space => less bit density => better reliability".

abc123abc123 3y ago
Amen! Exact same kodi-setup and sandisk cards, and has been happily running for years without problems. Disable all logs, media on smb/nfs, and off you go.
iforgotpassword 3y ago
I'm still running Kodi on a pi1 approaching 10 years of runtime on the original SD card. It's powered on most of the time but sometimes I accidentally power of the USB power supply it's connected to. It's a little 5-port one that's about just as old with a power button that's easy to press accidentally in my specific setup.
zikduruqe 3y ago
I have a RPi that has been running non-stop continuously since 2014 on the same SD card, serving up a weather website. I basically mounted all logging and webpages on tmpfs, and the DB resides on the SD card being written to every 5 minutes.
- seb1204 3y ago
  Are there instructions on how to upgrade my pihole to this?
  - zikduruqe 3y ago
    Start here. https://ostechnix.com/how-to-mount-a-temporary-partition-in-ram-in-linux/ https://ostechnix.com/how-to-mount-a-temporary-partition-in-... Just understand that anything that is written to tmpfs will be lost if you have a reboot. It might make troubleshooting difficult if you need to preserve them for whatever reason.

hcfman 3y ago

Another nice thing you can do if you use the multi-partitioned memory overlayFS approach I mentioned earlier is you can make /var/docker be a symlink to your read-write data partition. Obviously you are going to have problems using docker with the standard memory overlayFS approach. Also nice, is you can make your data or other partitions be encrypted. I've done this before. On the Pi 5 you can use the standard encryption as there's hardware support. On earlier Pi's you can use the encryption used for android. This does means there's a manual step in the startup for you to enter your encryption password.

planb 3y ago

Buy a decent SD card and overprovision the space and wear leveling will take care of the rest. I have a Pi with a 128GB SD card and a 32GB filesystem on it running for 6 years straight without problems now. No need to disable logs, just disable debug logging so you won‘t generate gigabytes a day.

hcfman 3y ago

Back in 2011 I made a commercial product that ran on the earliest plug computers from global scale technologies. I only sold 20 of 'em and every single one of them was being returned with SD card corruption problems. I had to quickly pivot to keeping the rootfs read-only. I've been a fan ever since. Incidentally, that early commercial product was a home security product with a very small amount of home automation. I released this into open source with a new name in 2021 and now runs on the Jetson series SBCs (https://github.com/hcfman/sbts-install https://github.com/hcfman/sbts-install). Except then including high end YOLO models as triggers. Because it was intended to be a standalone product it supported https with a GUI wrapper around all of the certificate operations. This still exists in my open source version, making it easy to use self signed certificates for intra-device rest calls. But I've kept and expanded upon the multi-partition memory overlayFS approach and the installation of this system first asks you to install the sbts-base system, which installs the multi-partition memory overlayFS so that other's can use this as their own base systems.

ikari_pl 3y ago
I had a similar experience when I hacked a $5 Pentium 2 PC into a fanless (whether it liked it or not) and noiseless workstation. Replaced the HDD with a CF card. After a while, the system started stalling for 1-2s on disk writes, and that was a pain.

MPSimmons 3y ago

I am urging everyone who wants to do this to see if they can't first do what they need with a small board like an ESP32. Their energy usage is a small fraction, they cost ones of dollars, and they're sufficient for a whole lot. If you're of the Python persuasion, many boards support both MicroPython and CircuitPython. It's worth looking into for the cost savings on initial purchase and ongoing power draw.

sofixa 3y ago
ESP32 and related are pretty cool, but it's a whole different mindset; if something doesn't work, you can't just connect an HDMI/keyboard to debug live with all the regular utilities one might know and that come for free with any mainiline Linux.
- MPSimmons 3y ago
  There are definitely limitations, but if it's a thing that you're going to leave untouched for years, it's worth looking at a device that will use 1/10th (or less) the power.
  - sofixa 3y ago
    Maybe, but if it is going to take you drastically longer to write the software because there's a smaller ecosystem / the stuff you need doesn't have a readymade library / you've never done C/MicroPython / there aren't the Linux tools that will help you debug or do a simple crontab to run your script on a schedule... The ROI might not be there even with the lower power consumption.
    - MPSimmons 3y ago
      That's true, and an important consideration. I'm not an ESP fanboy, but I really do appreciate it as an engineer for how simple and cheap the development boards are, especially considering things like ESPhome exist. It's kind of ridiculous how many projects can be built by just writing a yaml file and a little soldering - https://esphome.io/guides/diy.html https://esphome.io/guides/diy.html

tunnuz 3y ago

I wonder if Raspberry OS comes configured not to swap on the SD by default …

mvip 3y ago

We’ve been running thousands of Pis in production for about a decade now. We’re beginning to shift to x86. The price/performance isn’t what it once was for the Pi. I gave a talk about our experience recently at State of Open Con here (https://youtu.be/vX-qK9mxKZI https://youtu.be/vX-qK9mxKZI).

spintin 3y ago
You need to factor in usage, if you are idling a lot ARM > X86. And you need to look at longevity, there I suspect ARM will outlive X86 too. For modularity ARM > X86 too, because they are cheaper to have many small of. But for scalability (= business in the current economy) X86 > ARM. Also all graphs should be per watt, that 2 -> 4 is more performant is not news, that it is more performant per watt is! And if you did that you would see that Raspberry 5 is not getting as much per watt performance increase as it should. We have peaked permanently for the eternity of mankind. Let that sink in. Last but not least, the ONLY hope for any progress (openness not performance) at this point is the JH7110, but they are lagging behind in 3D support.
- mvip 3y ago
  Our customer don’t really care about power usage much as they are all connected to a TV which draws order of magnitude more power. For some use cases power usage is key, but not for our use case.
- AshamedCaptain 3y ago
  > if you are idling a lot ARM > X86. Don't take this for a given. RPis are notoriously bad at idle power consumption. The x86 replacement I bought for my home server ended up having about half the idle power consumption of the RPi it was replacing.
- pi-rat 3y ago
  I think maybe you’re overestimating the idle power consumption of modernish x86 (especially the last 5-6 gens of intel)? I’ve a i5 9600T system that’s drawing less than a Pi at idle. And cost me the same to buy. https://docs.google.com/spreadsheets/u/1/d/1LHvT2fRp7I6Hf18LcSzsNnjp10VI-odvwZpQZKv_NCI/htmlview#gid=0 https://docs.google.com/spreadsheets/u/1/d/1LHvT2fRp7I6Hf18L...

askvictor 3y ago

I've been running a Pi 4 as a home server for a few years. When boot-from-USB became available, I moved to that, with a good-quality USB thumb drive for boot/root (I've had had SD card issues in the past) (and also speed, though I doubt it makes much practical difference). A couple of weeks ago I started getting intermittent disk errors. I thought it was the USB drive, so I cloned it (still worked well enough on my laptop) to another drive. Same thing happened. So everything points to the USB controller glitching out. Have gone back to an SD card, and everything seems fine.

chopin 3y ago
Have a look at your power supply. I had intermittent file system errors with cheap ones. I threw them all out if only for the fire hazard.
- askvictor 3y ago
  Using the official pi 4 power supply.

cjdell 3y ago

I have an original Pi 2012 running buildroot. It's a perfect fit. Need just enough to run a Linux kernel and ser2net for doing RS485 stuff with solar inverters. I think the image size was around 100MB and no volatile filesystem whatsoever. Buildroot was surprisingly easy to use. Use a menuconfig to pick what you need and a burnable image for your SD card comes out the other side. Think I only spent an hour on the whole project.

archerx 3y ago

I have a cluster of 7 SBCs, 1 pi 3b and 6 Tinkerboards of various models. The Pi I got in 2016/17 and ran Gitlab without issue until 2022, something in the SD card got corrupt and linux was no longer able to boot, I was able to salvage all of the data and continued with a new SD card. The tinkerboards, which have various tasks as VPN, Nextcloud, staging servers using docker have never corrupted an SD card. I think the tinkerboards are better than Pi’s especially the ones that come with 16gb of onboard flash storage. However you don’t get all the niceness of PiOS but have to use TinkerOS (which was less than barebones when it came out) or Armbian which is nice but not built specifically for the tinkerboard. I have a few friends who complained about Pi’s corrupting SD cards and it also happened to my only long running Pi so there is something going on.

LtWorf 3y ago
I had an older rpi model without wifi where the network adapter got broken. Other than that, I've been running 3 rpi at home for home automation they have been working with no issue. But it turns out that they are a bit underpowered to be used as a minetest server. I never had a corrupted sd card on those. But I had an android nokia phone that corrupted like 3 sd cards before i gave up and stopped putting new ones.

spintin 3y ago

I have been running a Raspberry 2 cluster for 10 years. A few weeks back the first SD card to fail got so corrupted it failed to reboot! My key learning is use oversized cards, because then the bitcycling will wear slower! I'm going from 32GB to 256/512/1024! That said "High Endurance" cards are a scam, they fail way quicker than regular cards! All SD cards except SanDisk have latency problems. There is no competition. If you can get SLC SD cards use them for workload instances = no db or file storage.

exfil 3y ago

I run my RPi's 24/7 from initramfs. I can even remove card after boot.

greggsy 3y ago
Would you send them out to customers relying on them?
- exfil 3y ago
  Yes. They use them like that.

TacticalCoder 3y ago

Anything long running shouldn't happen over that piece of crap that WiFi is. Data centers aren't build over WiFi. My network at home neither. I've been running RPi for years (including a VoIP server on a RPi 1!). The two tricks if you want a really long running Pi are: SD cards mounted read-only and ethernet. FWIW I ve got a Pi running the unbound DNS resolver and it just works. It is not an art. There s a reason businesses are scooping up millions of Pi: they just work. P.S: I ve got an army of NUCs too.

LtWorf 3y ago
The ethernet card of my 1st rpi broke :D

RobotToaster 3y ago

Anything long running should have ECC RAM, which the Pi doesn't have.

anonym29 3y ago
This is only a concern for workloads where the loss of contents of memory is a significant problem beyond the need to reboot, no?

blackfawn 3y ago

Their suggestion for log2ram does help for the most part but depending on what the pi is running, even that doesn't completely solve it. I've burnt out a number of microSD cards running a Pi-hole instance. I finally gave up and moved it to a tiny x86 with a SSD. Yet I've been feeding ADSB-Exchange and FlightAware from a Pi Zero for years and never had SD card problems. I really like Alpine Linux for the Pi, running in it's own read-only mode where changes must be committed to disk. But unfortunately, Pi-hole isn't compatible with Alpine (at least last time I checked)

not_the_fda 3y ago

All the SD problems go away if you get a RPi Compute Module instead. Well worth the additional cost.

slow_typist 3y ago
How do the problems go away? Is the lifespan of the built in eMMC longer compared to a high quality wear leveling sd-card? I am a layman, but isn’t the problem the flash which should be basically the same?
- not_the_fda 3y ago
  An SD card is using contact pins to make the connection, which is susceptible to dirt and vibration. eMMC is soldered to the board so you no-longer need to worry about environmental and mechanical disturbances. The eMMC has a known supply chain and is guaranteed for 1 million write cycles that are then wear leveled. The SD card could be coming from anywhere, or a counterfeit with who knows how many write cycles and wear leveling. The CMs are built for industrial use and I've shipped a few products based on them, shipping thousands of units. They have had zero flash failures.
  - slow_typist 3y ago
    Thanks for elaborating, valuable advice.
  - megous 3y ago
    Not even NOR SLC flashes are regularly specified for 1 million erase cycles. And common MLC/TLC eMMCs certainly are not. What are you talking about? :)

hansonpeter 3y ago

[dead]

thehias 3y ago

And here is me, who is running a Pi1 in my cellar for 10 years straight, which logs all my temperature sensors over 433Mhz + triggers my door openers via physical relay over WIFI, without doing anything special. After some years I only connected it to a UPS, after the sdcard filesystem died after some power outages....

vagab0nd 3y ago

I use a USB connected SSD as boot. Been running Home Assistant and Pihole for years with zero issues. EDIT: Also, make sure the power supply is sufficient. I was using a cheap adapter and had random errors and reboots.

chopin 3y ago
Cheap adapters are a killer. I switched to a good 50W USB charger which powers a couple of Raspberries. Zero problems since then.

HankB99 3y ago

I use some Pis for various things in my house including Zeroes through CM4s and 4Bs. The Zeroes run Raspbian configured with the read-only filesystem option. I have found it necessary to uninstall `unattended-upgrades` because the overlayfs employed for read-only root caches disk writes in RAM and the update/upgrade process exhausts RAM. For the same reason I disable swap. It makes no sense to swap to RAM on a 512GB system. Upgrades are tedious since they require disabling overlayfs, rebooting, upgrading, rebooting, and enabling overlayfs. I wrote Ansible playbooks to perform these tasks. (https://github.com/HankB/Ansible/tree/main/Pi https://github.com/HankB/Ansible/tree/main/Pi) I have a Pi 4B performing as a file server and running Debian (not Raspbian) It boots from an SD card so that the entire HDDs can be used for a ZFS pool. To reduce wear and tear on the SD card I have mounted `/var` to a ZFS filesystem. I should probably use `tmpfs` for `/tmp`. I use a Pi CM4 to run HomeAssistant and that boots and runs from an NVME SSD where durability is less an issue.

lxgr 3y ago

> This is unlikely to do anything unless you’re hitting some unusual bug, but it’s worth noting that IPv6 has, in the past, led to all sorts of strange behaviors in different networking contexts. This makes me sad. I don't doubt that there are scenarios in which having IPv6 connectivity makes things worse, but these days, the opposite is more common, so I don't think "disable IPv6 just in case" is a good blanket recommendation to make anymore. "If disabling IPv6 fixes your issue consistently, consider disabling it" would achieve the same outcome, without potentially causing problems/inefficiencies down the road.

dannyw 3y ago
I agree with this. While I’ve had various issues with IPv6 a decade or so ago, nowadays I’ve never found disabling IPv6 to be a solution.

tgsovlerkhgsel 3y ago

I don't think I took many special precautions and a Pi I had running as a VPN server survived for about 10 years, aside from a power supply failure or two.

tzs 3y ago

A few people have mentioned achieving long uptime. What is often overlooked is that it is possible for uptime to be too long. It is quite possible for updates to not break a running system but make it so that it will break on the next reboot. E.g., a dynamic library gets updated in a way that breaks a server process. It doesn't affect the running server because it still has the old library loaded. Next time you boot your server process doesn't start. These kind of problems can be annoying to deal with, especially when your system has an uptime of years and for all you know whatever change broke it could have been in any one of dozens of updates you've applied over that time.

pmontra 3y ago
How do you deal with that? Probably one should have a canary system that is rebooted every day. In a home setup we usually don't hand either the spare machine or the spare time to deal with it, or both.
- tzs 3y ago
  The easiest way is to reboot after updates. Most updates are not urgent so unless you are having some problem that you hope the update will address wait until a time when it is OK to reboot. Then install the update and reboot. If there is an urgent update that you need to apply at a time when rebooting would be an issue apply the update and leave yourself a reminder that the system is running a configure that you don't know is bootable. When a reasonable time for a reboot comes around take it.
- progman32 3y ago
  Me personally, I always reboot after every update. I'd rather deal with the break now while I'm working on it, rather than at some random point. Then I'll know exactly what changed. I take it a step further. I usually reboot before updates, too. Make sure I won't chase updates as an issue if something else broke.

hn8305823 3y ago

> This feels like a hack, but based on hours of reading online discussions, most people seem to settle on a script that periodically checks whether the WiFi connection is good, and restarts the WiFi interface or the whole Pi if it’s not. It's not a hack, it's best practice! Just like important servers in a data center should have some kind of out of band connectivity (IPMI, remote controllable RPDU outlets, etc), Important servers in remote difficult to reach locations should have some kind of watchdog script. The script should of course be tuned to the specific use case, considering the impact of a reboot vs downtime until reboot. At the very least it could log adverse events for later investigation. A simple bash watchdog script was the very first thing I did when I deployed a remote RPI. Not just for wi-fi issues but for any of the dozens of things that could break and be fixed with a reboot.

vb6sp6 3y ago
[dead]
slicktux 3y ago
Totally agree! Watchdog timers are essential for microcontroller and even computers running software “forever”. Things do happen that even perfect code and design can’t prevent and a watchdog timer will break out of an infinite loop and reset. Things such as cosmic rays flipping a bit or even brownouts…in raspberry pi you also have to worry about SD card corruption… Edit: raspberry pi’s have built-in hardware watchdog timers I believe. I know Arduino's do!
bravetraveler 3y ago
Nowadays this watchdog is init/PID1, on most distributions - systemd. If init can't be relied on to manage services, what guarantees do you have for the system to provide them? Sure, one could reinvent this in scripts, but we've moved past this. I mention systemd a lot, but that's not to cast favor - there are alternatives. Most services don't make appropriate use of the environment they exist in. I assume they expect some site customization, ie: declaring your web server needs these mounts. A commonly overlooked directive is 'PartOf='. You can tie restarts of one service/resource to another. Heck, more simply, I think NetworkManager offers a way to customize the wifi/portal checking. You may not have to go completely heavy-handed

masto 3y ago

Step 1 is to question why you're using a Raspberry Pi. It's almost never the correct answer. If there's a really really good reason, step 2 is to get rid of the SD card. Personally, none of mine even have an SD card inserted (ok, one does). I use network boot/NFS for everything. Some people attach other kinds of SSDs. The one I lied about is a reverse telnet server that has been quietly doing its thing for 4 years without a hitch, apart from the time I had to replace the (PiJuice) backup battery because it was looking a little swole. But I should take the time to at least have a backup of the card ready to swap out when it fails.

bricee98 3y ago
I’m trying to make a smart speaker where you push a button to talk and when you release it sends the audio to a server for transcription and response. It also has to do some basic logic with LEDs. I also want it to be always on and available as long as it’s plugged in. Do you have any advice on what might be a better alternative to a Pi? This is basically my first foray into hardware so I’m trying to learn as much as possible!
ianburrell 3y ago
There are three uses for Pi. One is cheap computer, but Pi is never good at that and miniPCs have gotten cheap. Two is IoT, ESP32 is good choice for simple things and Pi Zero for bigger things. Three is for small servers and dedicated computers. Pi is perfect for that, it is cheaper than miniPCs, is smaller, and uses less power. Lots of people in this post have mentioned uses like that. I want to make outdoors ADS-B receiver, and Pi will fit in enclosure and be powered by PoE. I want to make GPS time server, and Pi has PPS input on header. I want to make portable ham radio box, and Pi can be powered by 12V DC.

cjfd 3y ago

My experience with long-running raspberry pi's is that there are USB problems. One sometimes loses the connection to these devices.

jokoon 3y ago

I don't understand what's hard about adding like 256MB of flash soldered to that thing, I'd pay $5 more if it did. Everytime there's a RPI I cry about it.

justin66 3y ago
Judging from a sampling of your comment history, every time someone points out that there's a Raspberry Pi compute module option that comes with flash memory.

greenie_beans 3y ago

eventually you'll think about power outages. i had a pihat battery nearly explode on me. beware!

anonym29 3y ago

Do people really have such a neurotic aversion to the sight of cables that they'd rather struggle with workarounds to an always-inferior wifi than just using an ethernet cable? It seems unhealthy to get that stressed out over the sight of a 3cm wide cable. I've had an RPi4 running continually for close to a year and never had a single network connectivity issue between the RPi4 and my router, connected via a 30m cat5 cable in my 75m² condo.

haunter 3y ago

I started buying Lenovo mini PCs instead, 18cm x 18cm x 3cm so it's still really small. And you can get them dirt cheap nowadays, has proper casing and cooling etc https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCentre_M720_Tiny/ThinkCentre_M720_Tiny_Spec.pdf https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCe... I have one right next to me, i5-8500T, 32GB RAM, 2x SSDs and currently 5W at idle with powertop auto-tune https://wiki.archlinux.org/title/powertop https://wiki.archlinux.org/title/powertop

thedougd 3y ago
Same. Bought a Dell refurb and I’ve had so many fewer hassles. The SD card was perhaps the biggest source of issues and after factoring the cost of outfitting a RPi with something like an SSD, it just wasn’t cost effective anymore.
- mbirth 3y ago
  This I never understood. I have 3 different RPis running since they were available to buy. A 2, a 3 and a 4. All with the OS on microSD and no optimisation like writing log files to tmpfs or similar. It’s plain Ubuntu Server for ARM. And in all these years, I had maybe one microSD going read-only on me. Cloned that to a fresh one, did an fsck and the Pi was up and running again. That’s it. No other issues despite various sudden power losses etc. I don’t understand where these rumours about bad reliability of SD cards in Pis come from.
  - thedougd 3y ago
    Maybe you're lucky or buying the right SD cards? I've always used noatime for filesystem, journald set to ram logging, etc. Usually everything seems fine until a reboot triggers a fsck. Most were ext 3/4 filesystems.
  - ecef9-8c0f-4374 3y ago
    Pi 1 a/b were really fragile but since pi 2 this problem is mostly solved. Still if I need to write a lot I would use a external drive. We have >300 pies running 24/7 on 52 sites all over the world. Even though we use no name brand microSDs we had only 2 failures because of a broken SD card in 4 years. Our partner company was using our hardware as a basis for something. After 1 month they asked us what SD cards brand we were using because they had a high failure rate. Our secret. We use a self made distro with minimal writes, they used default raspian and were writing permanently to the SD card.

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]

psanford 3y ago

Switching to gokrazy[0] was the best thing I did for my Raspberry Pi uptimes. I think a lot of that is because it defaults to using read-only partitions so the common issue of SD cards falling over when you run apt upgrade no longer happens. But I also think that gokrazy's simplicity and design helps it be just a solid, reliable foundation to build on top of. [0]: https://gokrazy.org/ https://gokrazy.org/

whitehexagon 3y ago
Looks interesting! but not clear to me who is behind the project? do you know if it is a commercial sponsorship, Golang team / Google, or pure community based open source ?
- psanford 3y ago
  There's no company behind it. Its a personal project by Michael Stapelberg.
NortySpock 3y ago
I used dietpi [1] for similar reasons: a slim version of Debian, and with the defaults set to push all the logging into ram to minimize writes. Dietpi has opinionated defaults, for sure, but it's easy to choose something else (e.g. Dropbear is the default ssh server, but bumping to OpenSSH is a matter of changing a setting in the handy config tool). I've been running an RPi3 on dietpi on an SD card as my secondary PiHole instance on it for at least a year with no issues. [1] https://dietpi.com/ https://dietpi.com/

nodesocket 3y ago

I can appreciate the effort and time put into this, but seems completely overkill. I’ve been running 4x Pi 4B’s in a Kubernetes cluster for over 3 years 24/7/365. Only thing I did is disable swap (which you should do for Kubernetes anyway) and used SanDisk Extreme 128GB cards[1]. [1] https://a.co/d/ipehodH https://a.co/d/ipehodH

deleted 3y ago

[deleted]

cattlepi 3y ago

Going to plug my own side project from the past here: https://cattlepi.com/ https://cattlepi.com/ https://github.com/cattlepi/cattlepi/blob/main/README.md https://github.com/cattlepi/cattlepi/blob/main/README.md Been running pis (mostly 3b+) for eons with this solution and at this point i can say it's bulletproof. The key is to minimize the sdcard wear and tear (it uses an overlay filesystem with squashfs as base and tmpfs as write top layer) and to keep zero stare on the device. You can build the image starting from normal raspbian. You can also update it over the network. As far as usage in the wild the largest "deployment" (i know about) is at around 1000 pis.

Zopieux 3y ago

Not a single mention of netbooting the Pi instead of relying on crappy SD cards which wear faster than I can type this.

vikmals 3y ago

> Your SD card can wear out or completely fill up Why is the author not considering using an SSD instead of an SD-Card here?

WaitWaitWha 3y ago

It is the the time for mini PCs as many have noted it. I am getting one for one of my offices, instead of building or buying a desktop. I used to build my machines and loved it. Some are still chugging along after over 10 years. They were worthwhile before the mini PCs reaching comparable capabilities at lower cost. One thing I still cannot seem to find is a good site that compares various vendors in similar fashion as "PC builder" sites do of build components. Any suggestions?

1vuio0pswjnm7 3y ago

"Keeping a Raspberry Pi online and working with zero intervention for weeks, months, or years is somewhat of an art form." I just boot NetBSD kernel with embedded filesystem, e.g., INSTALL kernel or custom kernel. SDCard can be removed immediately after boot. Optionally chroot to attached storage. This runs for weeks, months or years. Have not experienced any of the issues cited by the blog author. Only issue I find is with the power connector when using a case; the connection can be brittle, e.g., if using a replacement cable. Perhaps this has improved on more recent Pis. (But I could say the same about most computers. The cables and connectors are usually fragile. It's always cheap stuff.) If power is interrupted because of movement, then the Pi reboots automatically.

ChuckMcM 3y ago

The longest running RPi I have has run continuously for over 5 years. The big secret was not to use the SD card at all[1], I mount all file systems over the network to a NAS device (TruNAS from iX systems). It has a "UPS" in the form of a USB battery pack that is both charging from main power and powering the Pi. When power goes out the battery pack takes over, it has about a day of "hold up" time depending on Pi power usage. It is 'hard wired' to the local network (it doesn't use WiFi). I got there from having SD cards (nearly[2]) always be the failure point. Everything else has been pretty reliable when used within tolerances. [1] It does "boot" from the SD card but that acts kind of like a third stage bootloader which loads and boots the "real" OS (FreeBSD) from the NAS device. [2] I have had one fairly spectacular looking "melt down" of a no-name USB power supply wallwart PSU which, to appears to have also put something like 12V directly across the USB power pins (my best guess at what the secondary winding of the xformer in the wall wart was putting out on the 'low' side)

geon 3y ago

When I worked with that, we had the SD card entirely read-only, and a usb stick mounted for writing. I think the rpi wouold detect a broken file system on boot and reformat the usb stick. It also made it easy to pull the usb stick and format it in any windows pc to reset it.

husam212 3y ago

I use OpenWrt whenever it's possible, to avoid SD card wear completely.

pythonsnake 3y ago

Not sure about the SD cards he recommends. Use Swissbit SD cards. But they will cost you. Mouser is your friend.

demondemidi 3y ago

> Keeping a Raspberry Pi online and working with zero intervention for weeks, months, or years is somewhat of an art form. What are you talking about? It is literally zero effort. Just set up crontab reboots in case of power outage. That's it. I've had a pi running as a BLE gateway / security cam for over 4 years with zero intervention.

Animats 3y ago

Does that have a hardware stall timer?

jimmyswimmy 3y ago

Around early 2019 I set up a raspberry pi 3 running Raspbian. I made the /var/log partition a ramdisk. Haven't touched it since. It goes down for power outages but probably has been out for a couple minutes total over five years (aside from power outages). Most of its job is to translate analog audio to a USB speaker system. Whole house audio for about $150. Anyway, I never touch it, it just works, all the time.

jnaina 3y ago

I have a Raspberry PI 1 model B running almost non-stop for over 8 years (except during power outages) serving as my front gate controller. Custom python code uses the GPIO pins to trigger an RF remote to open or close the gate, and does not write to SD for logs/etc (to prevent SD wear) I use a USB network card, and a high-quality SD card. Other than that, no other special configs (except for another spare SD card with a full system image clone). Rock solid performance and uptime over 8 years.

rldjbpin 3y ago

having used an original model b (rev 2) since 2013 with running 24/7 for about 80% of the time, i feel that a lot of this depends on how much you want to push your pi. for context, my workload was bursty in nature and nowadays it is mostly idle. but i never had any issues with the SD card, and i only upgraded to a new one for getting more capacity all this time (only twice). keeping the dust out and managing the temperatures would go most of the way. i have served files directly from the sd card but it is always better to mount an external drive for this, while providing enough power to the board to power usb devices. limiting debug logs for stable applications can also help avoid write cycles, but using sd card on a pi has been a similar workload to using one in older smartphones for storing media.