6 ms·

> firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays t

by jnxx 6y ago

> firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels. > Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up. > power issue leading to corrupt "/" or any other SD card failure For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks. > usb disk breaking That's handled by RAID1 and backups. > OS crash borking everything I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system. > Raid might protect against a usbdisk breaking Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync. > its not even going to give you a performance boost either, its USB3 For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management. It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power-consumption-data/ https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power... > This is why n+1 is better So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

jnxx 6y ago
Just to complete this: As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging. Instead, I am using this script: https://github.com/fitu996/overlayRoot.sh https://github.com/fitu996/overlayRoot.sh which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.
KaiserPro 6y ago
> > OS crash borking everything > I've never seen that on Linux, in 22 years using it Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition. > So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB? Noooooope! just a second machine, with a copy of whatever you want to keep. when you are doing backups, have each one as a round robin target. Ceph is not something you'd want to run for redundancy/reliability. I've spent many years supporting "HA" filesystems. The simpler the better (unless its GPFS, that's actually good.)
- jnxx 6y ago
  > Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition. 1. That is what journaling file systems like ext4 are for. These are exactly to prevent that unfinished writes of metadata corrupt the file system. 2. Of course, it is possible that the Linux kernel goes all bonkers, writing random stuff to you disk and firing up nuclear missiles. But I have never seen that. I also have never seen a kernel Oops on a NAS. Kernel crashes are usually caused by faulty device drivers. 3. Also, in this aspect, there is no technical difference at all to a commercial NAS since they use Linux as well. They also use ARM CPUs, like the Raspberry, since they save power and do not need a fan (which in turns makes them less prone to failures). > just a second machine, with a copy of whatever you want to keep. Keeping backups is something I would seriously recomment for any kind of NAS or server. I do really not see how your argument applies in any way to a NAS based on a Raspberry Pi. The main difference I see is that it is substantially cheaper, and better suited to people who want to install extra software and occasionally like to tinker a bit. And I am obviously not recommending that the Central Bank of Canada or an institution like that uses it.

by jnxx 6y ago

jnxx 6y ago

Just to complete this: As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging. Instead, I am using this script: https://github.com/fitu996/overlayRoot.sh https://github.com/fitu996/overlayRoot.sh which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.

KaiserPro 6y ago

> > OS crash borking everything > I've never seen that on Linux, in 22 years using it Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition. > So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB? Noooooope! just a second machine, with a copy of whatever you want to keep. when you are doing backups, have each one as a round robin target. Ceph is not something you'd want to run for redundancy/reliability. I've spent many years supporting "HA" filesystems. The simpler the better (unless its GPFS, that's actually good.)

jnxx 6y ago
> Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition. 1. That is what journaling file systems like ext4 are for. These are exactly to prevent that unfinished writes of metadata corrupt the file system. 2. Of course, it is possible that the Linux kernel goes all bonkers, writing random stuff to you disk and firing up nuclear missiles. But I have never seen that. I also have never seen a kernel Oops on a NAS. Kernel crashes are usually caused by faulty device drivers. 3. Also, in this aspect, there is no technical difference at all to a commercial NAS since they use Linux as well. They also use ARM CPUs, like the Raspberry, since they save power and do not need a fan (which in turns makes them less prone to failures). > just a second machine, with a copy of whatever you want to keep. Keeping backups is something I would seriously recomment for any kind of NAS or server. I do really not see how your argument applies in any way to a NAS based on a Raspberry Pi. The main difference I see is that it is substantially cheaper, and better suited to people who want to install extra software and occasionally like to tinker a bit. And I am obviously not recommending that the Central Bank of Canada or an institution like that uses it.