5 ms·

I just spent about a week trying to get a raidz pool going for a home server under Ubuntu. It was about one of the most frustrating experiences in recent memory

by xattt 7y ago

I just spent about a week trying to get a raidz pool going for a home server under Ubuntu. It was about one of the most frustrating experiences in recent memory. My goal was to run the root system off an SSD with the heavily used folders offloaded onto a ZFS raidz pool. I followed the Ubuntu Root on ZFS guide with changes I thought would be appropriate. I gave up eventually and came to a disgruntled conclusion is that it works for beginner users tho follow guides to the T and advanced users that know the changes that need to be made from institutional knowledge, but not for people in between.

Proven 7y ago
But you can say you're cool because you use ZFS... Some people no doubt need it, most don't. It's probably cheaper to buy one large disk and regularly back up it's essential contents to S3.
Filligree 7y ago
That depends on your OS. Ubuntu is, frankly, very bad at it -- though few other distributions are very much better. I'd recommend NixOS. ZFS is a proper first-class filesystem there, and you can use it almost however you like.
- xattt 7y ago
  Great, thank you. I will look into it.
- tracker1 7y ago
  Thanks... have a used server I'm going to play with in a couple weekends... won't have the final drives for the new shared drive setup for a couple months (moving away from an old nas box) and going to try a few scenarios with an nvme as the boot drive via adapter, and most storage/backup to the RAID array. Was planning on trying both unraid and windows server as host OSes, and maybe even using a NAS distro in a VM managing the individual drives, etc. Would prefer something that supported docker, full vms and a friendly NAS UI, but not sure such a beast exists.
  - dflock 7y ago
    FreeNAS is linux distro with a friendly NAS UI: https://freenas.org/ https://freenas.org/ - but still linux, so you get all the usual linux stuff - docker, full VMs, etc...
    - chousuke 7y ago
      FreeNAS is based on FreeBSD, not the Linux kernel. It's from a completely different lineage of operating systems.
      - dflock 7y ago
        Ooops, thanks for the correction!
    - tracker1 7y ago
      As mentioned in another comment, FreeNAS is based on FreeBSD and they dropped Linux/Docker compatibility, and full VMs are less similar to other platforms. If I did FreeNAS, it would be in a VM with the drives mapped to the VM.
      - dflock 7y ago
        Ooops, I thought it was Linux, sorry!
radiowave 7y ago
Yes. I tried this maybe a year or so back, and came to much the same conclusion, and I say this as someone who's been running ZFS systems for over a decade now. When it comes to actually booting from them, the potential for "fit and finish" issues with the distro (and grub...) is very significant. Proxmox works really well for me, you can basically treat it as a non-desktop Debian distro, with ZFS built-in, and a largish community of people who use it that way.
kogir 7y ago
What exactly did you try? On the surface that sounds pretty straightforward. The problem I most often see people run into is trying to use mismatched drives they have lying around. With ZFS you’ll have a much better time just purchasing matched disks to fit the zpools you have in mind. It was built for well planned, huge arrays in datacenters and that legacy shows. I’ve used ZFS for years in a variety of configurations on Solaris, FreeBSD and macOS. My only training was the Solaris and FreeBSD man pages, so it’s not that dark an art.
- JustSomeNobody 7y ago
  So, what does one do if all they have laying around are mismatched drives?
  - meroje 7y ago
    You can partition your drives based on the size of the smallest and make your pool on those partitions.
  - magicalhippo 7y ago
    You _can_ mix drives, but you'll get the worst from each. For each vdev (mirror pair, or raid stripe) writes will be limited by the slowest drive, and space will be limited by the smallest drive. So if they're not highly mismatched, it can work ok. For example if you have a 1 TB and a 1.5 TB disk, but from the same series and as such with otherwise similar spec, it would work fine as a mirror vdev. Write speed will be limited to the slowest of them (most likely the 1 TB), and you'd only get 1 TB of available space, but it's more than nothing.
    - JustSomeNobody 7y ago
      Ah, ok. I have 2 3TB and a 4TB running on a Proxmox v3 system. Was wanting to upgrade to Proxmox v5 and try ZFS. The 4TB is a different make than the 2 3TB drives, though.
      - 7y ago

by xattt 7y ago

Proven 7y ago

But you can say you're cool because you use ZFS... Some people no doubt need it, most don't. It's probably cheaper to buy one large disk and regularly back up it's essential contents to S3.

Filligree 7y ago

That depends on your OS. Ubuntu is, frankly, very bad at it -- though few other distributions are very much better. I'd recommend NixOS. ZFS is a proper first-class filesystem there, and you can use it almost however you like.

xattt 7y ago
Great, thank you. I will look into it.
tracker1 7y ago
Thanks... have a used server I'm going to play with in a couple weekends... won't have the final drives for the new shared drive setup for a couple months (moving away from an old nas box) and going to try a few scenarios with an nvme as the boot drive via adapter, and most storage/backup to the RAID array. Was planning on trying both unraid and windows server as host OSes, and maybe even using a NAS distro in a VM managing the individual drives, etc. Would prefer something that supported docker, full vms and a friendly NAS UI, but not sure such a beast exists.
- dflock 7y ago
  FreeNAS is linux distro with a friendly NAS UI: https://freenas.org/ https://freenas.org/ - but still linux, so you get all the usual linux stuff - docker, full VMs, etc...
  - chousuke 7y ago
    FreeNAS is based on FreeBSD, not the Linux kernel. It's from a completely different lineage of operating systems.
    - dflock 7y ago
      Ooops, thanks for the correction!
  - tracker1 7y ago
    As mentioned in another comment, FreeNAS is based on FreeBSD and they dropped Linux/Docker compatibility, and full VMs are less similar to other platforms. If I did FreeNAS, it would be in a VM with the drives mapped to the VM.
    - dflock 7y ago
      Ooops, I thought it was Linux, sorry!

radiowave 7y ago

Yes. I tried this maybe a year or so back, and came to much the same conclusion, and I say this as someone who's been running ZFS systems for over a decade now. When it comes to actually booting from them, the potential for "fit and finish" issues with the distro (and grub...) is very significant. Proxmox works really well for me, you can basically treat it as a non-desktop Debian distro, with ZFS built-in, and a largish community of people who use it that way.

kogir 7y ago

What exactly did you try? On the surface that sounds pretty straightforward. The problem I most often see people run into is trying to use mismatched drives they have lying around. With ZFS you’ll have a much better time just purchasing matched disks to fit the zpools you have in mind. It was built for well planned, huge arrays in datacenters and that legacy shows. I’ve used ZFS for years in a variety of configurations on Solaris, FreeBSD and macOS. My only training was the Solaris and FreeBSD man pages, so it’s not that dark an art.

JustSomeNobody 7y ago
So, what does one do if all they have laying around are mismatched drives?
- meroje 7y ago
  You can partition your drives based on the size of the smallest and make your pool on those partitions.
- magicalhippo 7y ago
  You _can_ mix drives, but you'll get the worst from each. For each vdev (mirror pair, or raid stripe) writes will be limited by the slowest drive, and space will be limited by the smallest drive. So if they're not highly mismatched, it can work ok. For example if you have a 1 TB and a 1.5 TB disk, but from the same series and as such with otherwise similar spec, it would work fine as a mirror vdev. Write speed will be limited to the slowest of them (most likely the 1 TB), and you'd only get 1 TB of available space, but it's more than nothing.
  - JustSomeNobody 7y ago
    Ah, ok. I have 2 3TB and a 4TB running on a Proxmox v3 system. Was wanting to upgrade to Proxmox v5 and try ZFS. The 4TB is a different make than the 2 3TB drives, though.
    - 7y ago