4 ms·

what ever you do, dont do this. The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep. if you want to us

what ever you do, dont do this. The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep. if you want to use a raspberry pi for network storage, and you want to keep your data, make sure you have n+1 pi & disk machine. Raid isn't going to help you here, you need redundant nodes. for the same €240 you could get two minimum spec pi4s PSU and usb HDDs

jnxx 6y ago
> make sure you have n+1 pi & disk machine. Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL). The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks. > Raid isn't going to help you here, you need redundant nodes. Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?
- KaiserPro 6y ago
  > Why exactly? My set-up is working well congratulations I am very happy for you! The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much. You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. > Raid isn't going to help you here, you need redundant nodes. This setup will have four common failure modes: o power issue leading to corrupt "/" or any other SD card failure o usb disk breaking o OS crash borking everything o Accidental deletion Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3. This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.
  - jnxx 6y ago
    > firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels. > Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up. > power issue leading to corrupt "/" or any other SD card failure For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks. > usb disk breaking That's handled by RAID1 and backups. > OS crash borking everything I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system. > Raid might protect against a usbdisk breaking Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync. > its not even going to give you a performance boost either, its USB3 For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management. It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power-consumption-data/ https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power... > This is why n+1 is better So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?
    - jnxx 6y ago
      Just to complete this: As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging. Instead, I am using this script: https://github.com/fitu996/overlayRoot.sh https://github.com/fitu996/overlayRoot.sh which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.

jnxx 6y ago

> make sure you have n+1 pi & disk machine. Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL). The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks. > Raid isn't going to help you here, you need redundant nodes. Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?

KaiserPro 6y ago
> Why exactly? My set-up is working well congratulations I am very happy for you! The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much. You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. > Raid isn't going to help you here, you need redundant nodes. This setup will have four common failure modes: o power issue leading to corrupt "/" or any other SD card failure o usb disk breaking o OS crash borking everything o Accidental deletion Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3. This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.
- jnxx 6y ago
  > firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels. > Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up. > power issue leading to corrupt "/" or any other SD card failure For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks. > usb disk breaking That's handled by RAID1 and backups. > OS crash borking everything I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system. > Raid might protect against a usbdisk breaking Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync. > its not even going to give you a performance boost either, its USB3 For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management. It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power-consumption-data/ https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power... > This is why n+1 is better So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?
  - jnxx 6y ago
    Just to complete this: As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging. Instead, I am using this script: https://github.com/fitu996/overlayRoot.sh https://github.com/fitu996/overlayRoot.sh which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.