6 ms·

What's a cheap option for a ZFS based NAS systme at home? QNAP is now offering ZFS based systems but they are rather pricey. Is it just BYO? How can i make one

by Tepix 6y ago

What's a cheap option for a ZFS based NAS systme at home? QNAP is now offering ZFS based systems but they are rather pricey. Is it just BYO? How can i make one that is super power efficient?

Certified 6y ago
I highly recommend the BYO route, especially if you have part of a suitable machine already lying around. Just about any x86 based system that can take 16GB+ of memory will be a good candidate. ECC is recommended but many people, myself included, have run many stable years without ECC. Load the latest Ubuntu up and give this [1] ars how-to article a read (keeping in mind it is several years old now). Power efficiency will mostly hinge on the cpu and how many hard drives you keep spinning. To scale past the the number of SATA ports you have on a motherboard, add an LSI HBA card like the 9300-8E or 9207-8E. [1] https://arstechnica.com/information-technology/2014/02/ars-walkthrough-using-the-zfs-next-gen-filesystem-on-linux/ https://arstechnica.com/information-technology/2014/02/ars-w...
- Osiris 6y ago
  I bought a mini-ITX board with 4 SATA ports and an embedded CPU. I added a PCIe card with 2 additional SATA ports. I bought a mini-ITX case with mounts for 6 HDDs. I have 5 8TB drives (RAIDZ2) and 1 SSD as the boot drive. The system runs Ubuntu Server. I found this setup a lot cheaper than the dedicated NAS boxes you can buy and more flexible.
  - vermaden 6y ago
    Can you post a link to that Mini-ITX case? Thanks.
    - jtbarrett 6y ago
      I’m not the parent but I built something similar in the Fractal Node 304 which supports six 2.5” drives. Mine’s been going strong for 7 years now. https://www.fractal-design.com/products/cases/node/node-304 https://www.fractal-design.com/products/cases/node/node-304
      - Osiris 6y ago
        That's exactly the one I have.
      - vermaden 6y ago
        Thanks.
  - znpy 6y ago
    Yes please, can you post the brand/model of that case?
- wting 6y ago
  >ECC is recommended but many people, myself included, have run many stable years without ECC. zfs requires ECC[0][1], do not skimp on this. I ran many zfs pools for >8 years without issues until silent memory corruption happened to me, and now I can't access my data anymore (zfs won't let me mount a zpool that fails scrub checks). 0: Oracle (and previously Sun) documentation recommends ECC to prevent silent corruption: >Consider using ECC memory to protect against memory corruption. Silent memory corruption can potentially damage your data. https://docs.oracle.com/cd/E23824_01/html/821-1448/zfspools-4.html https://docs.oracle.com/cd/E23824_01/html/821-1448/zfspools-... 1: https://louwrentius.com/please-use-zfs-with-ecc-memory.html https://louwrentius.com/please-use-zfs-with-ecc-memory.html
  - vermaden 6y ago
    Please stop repeating this lie that ZFS somehow needs ECC more then other filesystems. All filesystems benefit from ECC memory but ZFS from all filesystems does its job best even without ECC. Page 102 from https://is.gd/bsdstg https://is.gd/bsdstg here. I'm getting bored repeating this over and over again.
    - GauntletWizard 6y ago
      The problem isn’t that ZFS needs ECC more. It’s that other filesystems will silently ignore the errors that ZFS points out. With other filesystems, you data just eats some corruption. With ZFS, you’re likely to encounter errors because you notice them - Because it’s extensive checksumming points out when a bit flips. I’d much prefer to know, but if you’re used to not knowing, knowing is scary.
  - agapon 6y ago
    As the sibling comment says, ZFS does not _require_ ECC. It's your data that requires it. Whichever filesystem you use. If you care about the data, of course.
magicalhippo 6y ago
Since you mention QNAP, an alternative if you're mostly just looking for a NAS then an old computer with TrueNAS (formerly FreeNAS) is a nice alternative. You don't need more than about 4GB of RAM, however ZFS is quite good at using it for cache so the more the better. Anyway, whatever you do, don't enable deduplication. You can't simply turn it off and it imposes sever limitations.
- znpy 6y ago
  You can do with even less ram... But you should really throw all the ram you can at zfs. It'll just make performance better, and you'll hit the disks way less (for read operations).
quux 6y ago
I have a Helios64 NAS that I'm running a raidz pool of 5 3tb drives on with no issues. Full kit of board and enclosure was $295. Overall I've been very happy with it. The only negative from a ZFS perspective is it doesn't have enough ram to support deduplication but that's not a big issue for my use case and otherwise I've been very happy with it. https://kobol.io/ https://kobol.io/
- livearchivist 6y ago
  What kernel are you on and what steps did you take to get ZFS running? I can't seem to get it up and running.
  - quux 6y ago
    I’m using the latest 5.9 kernel on Debian buster. I had to compile from source and make a small patch to get the build working. Happy to share a gist. I wanted to use ads 2.0 but if you want something g mower friction use Ubuntu focal I think there are packages already in the distro
- abrookewood 6y ago
  The helios64 looks so damn cool ... I'm currently running an aging HP Microsoerver which has served me well, but I think the Helios will be my next build.
- doublepg23 6y ago
  ZFS dedupe is effectively broken and I wouldn’t recommend anyone use it unless you really know what you’re getting into.
vermaden 6y ago
For example something like that: https://vermaden.wordpress.com/2019/04/03/silent-fanless-freebsd-server-redundant-backup/ https://vermaden.wordpress.com/2019/04/03/silent-fanless-fre... It will provide you a ZFS mirror of two 5 TB 2.5 drives. I repaste the table from the article for convenience. PRICE TIME SERVICE $447 5 YEAR(S) Self Build NAS $567 5 YEAR(S) Self Build NAS (assuming one of the drives failed) $1500 5 YEAR(S) Amazon Drive $1550 5 YEAR(S) Google One $1200 5 YEAR(S) Amazon S3 Glacier Storage $2250 5 YEAR(S) Backblaze B2 Cloud Storage
- prirun 6y ago
  The Backblaze B2 prices are not correct. In the original article, it states: "For the Backblaze B2 Cloud Storage I assumed average between upload/download price because upload is two times cheaper then download." In fact, uploads on B2 are free, and download bandwidth is cheaper than any other supplier I know of at 1 cent per GB, with a free daily allowance of 1GB. Storage rates are 1/2 cent per GB per month, of $5/mo per TB, including redundancy. The storage cost of 5TB for Backblaze B2 for 5 years would be $60/year or $300 for 5 years, the upload cost is free, and if you downloaded your entire 5TB each month, it would cost less than $600 over 5 years because you'd also have the 1GB/day free download allowance. HashBackup (I'm the author) uses the free B2 daily allowance to pack older backup data, which sometimes requires an download and upload cycle. I couldn't find the upload/download bandwidth assumptions, but these numbers are way off IMO.
geek_at 6y ago
cheap hardware with fresh disks + Alpine Linus is my personal preference. Even with full LUKS disk encryption https://blog.haschek.at/2020/the-perfect-file-server.html https://blog.haschek.at/2020/the-perfect-file-server.html
mschuster91 6y ago
I went and picked up an old dual-CPU Mac Pro 4.1 - four disk slots with WD Red 4TB and one SSD in the 2nd ODD slot for Debian. It is extremely silent after cleaning the fans and replacing the aged GPU with a silent one but not power efficient, that's granted. But at least it's having actual processing power and can sustain ~120MB/s write unlike its predecessor (Lenovo ix4-300d) that could barely be bothered to do 10 MB/s...

by Tepix 6y ago

What's a cheap option for a ZFS based NAS systme at home? QNAP is now offering ZFS based systems but they are rather pricey. Is it just BYO? How can i make one that is super power efficient?

Certified 6y ago

I highly recommend the BYO route, especially if you have part of a suitable machine already lying around. Just about any x86 based system that can take 16GB+ of memory will be a good candidate. ECC is recommended but many people, myself included, have run many stable years without ECC. Load the latest Ubuntu up and give this [1] ars how-to article a read (keeping in mind it is several years old now). Power efficiency will mostly hinge on the cpu and how many hard drives you keep spinning. To scale past the the number of SATA ports you have on a motherboard, add an LSI HBA card like the 9300-8E or 9207-8E. [1] https://arstechnica.com/information-technology/2014/02/ars-walkthrough-using-the-zfs-next-gen-filesystem-on-linux/ https://arstechnica.com/information-technology/2014/02/ars-w...

Osiris 6y ago
I bought a mini-ITX board with 4 SATA ports and an embedded CPU. I added a PCIe card with 2 additional SATA ports. I bought a mini-ITX case with mounts for 6 HDDs. I have 5 8TB drives (RAIDZ2) and 1 SSD as the boot drive. The system runs Ubuntu Server. I found this setup a lot cheaper than the dedicated NAS boxes you can buy and more flexible.
- vermaden 6y ago
  Can you post a link to that Mini-ITX case? Thanks.
  - jtbarrett 6y ago
    I’m not the parent but I built something similar in the Fractal Node 304 which supports six 2.5” drives. Mine’s been going strong for 7 years now. https://www.fractal-design.com/products/cases/node/node-304 https://www.fractal-design.com/products/cases/node/node-304
    - Osiris 6y ago
      That's exactly the one I have.
    - vermaden 6y ago
      Thanks.
- znpy 6y ago
  Yes please, can you post the brand/model of that case?
wting 6y ago
>ECC is recommended but many people, myself included, have run many stable years without ECC. zfs requires ECC[0][1], do not skimp on this. I ran many zfs pools for >8 years without issues until silent memory corruption happened to me, and now I can't access my data anymore (zfs won't let me mount a zpool that fails scrub checks). 0: Oracle (and previously Sun) documentation recommends ECC to prevent silent corruption: >Consider using ECC memory to protect against memory corruption. Silent memory corruption can potentially damage your data. https://docs.oracle.com/cd/E23824_01/html/821-1448/zfspools-4.html https://docs.oracle.com/cd/E23824_01/html/821-1448/zfspools-... 1: https://louwrentius.com/please-use-zfs-with-ecc-memory.html https://louwrentius.com/please-use-zfs-with-ecc-memory.html
- vermaden 6y ago
  Please stop repeating this lie that ZFS somehow needs ECC more then other filesystems. All filesystems benefit from ECC memory but ZFS from all filesystems does its job best even without ECC. Page 102 from https://is.gd/bsdstg https://is.gd/bsdstg here. I'm getting bored repeating this over and over again.
  - GauntletWizard 6y ago
    The problem isn’t that ZFS needs ECC more. It’s that other filesystems will silently ignore the errors that ZFS points out. With other filesystems, you data just eats some corruption. With ZFS, you’re likely to encounter errors because you notice them - Because it’s extensive checksumming points out when a bit flips. I’d much prefer to know, but if you’re used to not knowing, knowing is scary.
- agapon 6y ago
  As the sibling comment says, ZFS does not _require_ ECC. It's your data that requires it. Whichever filesystem you use. If you care about the data, of course.

magicalhippo 6y ago

Since you mention QNAP, an alternative if you're mostly just looking for a NAS then an old computer with TrueNAS (formerly FreeNAS) is a nice alternative. You don't need more than about 4GB of RAM, however ZFS is quite good at using it for cache so the more the better. Anyway, whatever you do, don't enable deduplication. You can't simply turn it off and it imposes sever limitations.

znpy 6y ago
You can do with even less ram... But you should really throw all the ram you can at zfs. It'll just make performance better, and you'll hit the disks way less (for read operations).

quux 6y ago

I have a Helios64 NAS that I'm running a raidz pool of 5 3tb drives on with no issues. Full kit of board and enclosure was $295. Overall I've been very happy with it. The only negative from a ZFS perspective is it doesn't have enough ram to support deduplication but that's not a big issue for my use case and otherwise I've been very happy with it. https://kobol.io/ https://kobol.io/

livearchivist 6y ago
What kernel are you on and what steps did you take to get ZFS running? I can't seem to get it up and running.
- quux 6y ago
  I’m using the latest 5.9 kernel on Debian buster. I had to compile from source and make a small patch to get the build working. Happy to share a gist. I wanted to use ads 2.0 but if you want something g mower friction use Ubuntu focal I think there are packages already in the distro
abrookewood 6y ago
The helios64 looks so damn cool ... I'm currently running an aging HP Microsoerver which has served me well, but I think the Helios will be my next build.
doublepg23 6y ago
ZFS dedupe is effectively broken and I wouldn’t recommend anyone use it unless you really know what you’re getting into.

vermaden 6y ago

For example something like that: https://vermaden.wordpress.com/2019/04/03/silent-fanless-freebsd-server-redundant-backup/ https://vermaden.wordpress.com/2019/04/03/silent-fanless-fre... It will provide you a ZFS mirror of two 5 TB 2.5 drives. I repaste the table from the article for convenience. PRICE TIME SERVICE $447 5 YEAR(S) Self Build NAS $567 5 YEAR(S) Self Build NAS (assuming one of the drives failed) $1500 5 YEAR(S) Amazon Drive $1550 5 YEAR(S) Google One $1200 5 YEAR(S) Amazon S3 Glacier Storage $2250 5 YEAR(S) Backblaze B2 Cloud Storage

prirun 6y ago
The Backblaze B2 prices are not correct. In the original article, it states: "For the Backblaze B2 Cloud Storage I assumed average between upload/download price because upload is two times cheaper then download." In fact, uploads on B2 are free, and download bandwidth is cheaper than any other supplier I know of at 1 cent per GB, with a free daily allowance of 1GB. Storage rates are 1/2 cent per GB per month, of $5/mo per TB, including redundancy. The storage cost of 5TB for Backblaze B2 for 5 years would be $60/year or $300 for 5 years, the upload cost is free, and if you downloaded your entire 5TB each month, it would cost less than $600 over 5 years because you'd also have the 1GB/day free download allowance. HashBackup (I'm the author) uses the free B2 daily allowance to pack older backup data, which sometimes requires an download and upload cycle. I couldn't find the upload/download bandwidth assumptions, but these numbers are way off IMO.

geek_at 6y ago

cheap hardware with fresh disks + Alpine Linus is my personal preference. Even with full LUKS disk encryption https://blog.haschek.at/2020/the-perfect-file-server.html https://blog.haschek.at/2020/the-perfect-file-server.html

mschuster91 6y ago

I went and picked up an old dual-CPU Mac Pro 4.1 - four disk slots with WD Red 4TB and one SSD in the 2nd ODD slot for Debian. It is extremely silent after cleaning the fans and replacing the aged GPU with a silent one but not power efficient, that's granted. But at least it's having actual processing power and can sustain ~120MB/s write unlike its predecessor (Lenovo ix4-300d) that could barely be bothered to do 10 MB/s...