3 ms·

You can use a Raspberry Pi (perhaps a Pi 400) if you work around the fact that the internal flash memory is not made for constant writing. Thus, you have to arr

by jnxx 6y ago

You can use a Raspberry Pi (perhaps a Pi 400) if you work around the fact that the internal flash memory is not made for constant writing. Thus, you have to arrange the system so that it is read-only in normal state - which is not difficult. The Pi 400 also has an advantage that it has a keyboard, so you can cleanly shut it down without logging in, which makes a lot of sense for a storage device. Also, for RAID1 you need an extra powered USB hub. The normal power supply of the Pi 400 is not sufficient for more than one disk. Then just install a software RAID, NFS and possibly rsync and Samba. A simple NAS built in this way will come at about 240€, depending on disk size and will be mostly equivalent to a commercial one for twice the price. As an advantage, you can run well-known, standard software which is completely under your control. And since it is Debian-based Raspbian, it can be upgraded easily without breaking stuff.

KaiserPro 6y ago
what ever you do, dont do this. The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep. if you want to use a raspberry pi for network storage, and you want to keep your data, make sure you have n+1 pi & disk machine. Raid isn't going to help you here, you need redundant nodes. for the same €240 you could get two minimum spec pi4s PSU and usb HDDs
- jnxx 6y ago
  > make sure you have n+1 pi & disk machine. Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL). The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks. > Raid isn't going to help you here, you need redundant nodes. Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?
  - KaiserPro 6y ago
    > Why exactly? My set-up is working well congratulations I am very happy for you! The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much. You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. > Raid isn't going to help you here, you need redundant nodes. This setup will have four common failure modes: o power issue leading to corrupt "/" or any other SD card failure o usb disk breaking o OS crash borking everything o Accidental deletion Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3. This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.
    - jnxx 6y ago
      > firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels. > Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up. > power issue leading to corrupt "/" or any other SD card failure For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks. > usb disk breaking That's handled by RAID1 and backups. > OS crash borking everything I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system. > Raid might protect against a usbdisk breaking Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync. > its not even going to give you a performance boost either, its USB3 For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management. It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power-consumption-data/ https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power... > This is why n+1 is better So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?
sokoloff 6y ago
Clean shutdowns are indeed preferred, but synology devices will do a clean shutdown (delayed as required) if you press and hold the power button until it beeps and starts flashing, then release.
- jnxx 6y ago
  Yeah. That is a problem with older Raspberry Pis, which do not have a keyboard. It is possible to use a GPIO port for this, and shut it down with an attached button. But one has to be careful not to use the port unless it is configured properly, because that could cause a short-circuit, and the R-Pi is not protected against that. Having a keyboard on which you can type Ctrl-Alt-Del is much easier to use and will work on most relevant circumstances. In modern Raspian editions, this can be configured in systemd.
jnxx 6y ago
I'd be really curious to know why people are downvoting this. I can't see any technical reason against it. Of course, for some people this might be too complicated or too much hassle in comparison to buying a more expensive commercial thing. But for other people, the cost savings might be worth the work. It is also easier to keep an almost completely open source solution fully up-to-date.

by jnxx 6y ago

KaiserPro 6y ago

what ever you do, dont do this. The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep. if you want to use a raspberry pi for network storage, and you want to keep your data, make sure you have n+1 pi & disk machine. Raid isn't going to help you here, you need redundant nodes. for the same €240 you could get two minimum spec pi4s PSU and usb HDDs

jnxx 6y ago
> make sure you have n+1 pi & disk machine. Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL). The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks. > Raid isn't going to help you here, you need redundant nodes. Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?
- KaiserPro 6y ago
  > Why exactly? My set-up is working well congratulations I am very happy for you! The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much. You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. > Raid isn't going to help you here, you need redundant nodes. This setup will have four common failure modes: o power issue leading to corrupt "/" or any other SD card failure o usb disk breaking o OS crash borking everything o Accidental deletion Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3. This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.
  - jnxx 6y ago
    > firstly you are reliant on the disks presenting themselves in the same order at the same time. In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels. > Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile. No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up. > power issue leading to corrupt "/" or any other SD card failure For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks. > usb disk breaking That's handled by RAID1 and backups. > OS crash borking everything I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system. > Raid might protect against a usbdisk breaking Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync. > its not even going to give you a performance boost either, its USB3 For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management. It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power-consumption-data/ https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power... > This is why n+1 is better So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

sokoloff 6y ago

Clean shutdowns are indeed preferred, but synology devices will do a clean shutdown (delayed as required) if you press and hold the power button until it beeps and starts flashing, then release.

jnxx 6y ago
Yeah. That is a problem with older Raspberry Pis, which do not have a keyboard. It is possible to use a GPIO port for this, and shut it down with an attached button. But one has to be careful not to use the port unless it is configured properly, because that could cause a short-circuit, and the R-Pi is not protected against that. Having a keyboard on which you can type Ctrl-Alt-Del is much easier to use and will work on most relevant circumstances. In modern Raspian editions, this can be configured in systemd.

jnxx 6y ago

I'd be really curious to know why people are downvoting this. I can't see any technical reason against it. Of course, for some people this might be too complicated or too much hassle in comparison to buying a more expensive commercial thing. But for other people, the cost savings might be worth the work. It is also easier to keep an almost completely open source solution fully up-to-date.