4 ms·

If you only need a single disk filesystem, it's basically the same plus some minor RAM hunger (doesn't run well with 1-2GB regardless of how many disks you have

by zaarn 8y ago

If you only need a single disk filesystem, it's basically the same plus some minor RAM hunger (doesn't run well with 1-2GB regardless of how many disks you have). ZFS isn't quite easier to user for multiple disks or volumes, rather, there is a very tiny set of configurations that everyone recommends (RAID1 mirrors, leaving consumers unable to grow pools 1 disk at a time because ZFS is fundamentally unable to grow a RAID vdev). LVM and mdadm manage these jobs much better and are composable to your needs.

laumars 8y ago
> doesn't run well with 1-2GB regardless of how many disks you have I've read people say that but I've never had any issues when I used to run ZFS on various systems with 2GB RAM. So I wonder how many of those people actually had practical experiences running ZFS on low memory systems and how much of it is people just looking at some of the recommended memory specifications (which are high) and start making assumptions about it's performance on low memory systems. > RAID1 mirrors, leaving consumers unable to grow pools 1 disk at a time because ZFS is fundamentally unable to grow a RAID vdev You actually can grow a ZFS vdev - exactly by replacing one disk at a time (albeit you don't get the gained capacity until all the disks in the vdev have been replaced). What you cannot do is shrink it - but that's not a unique problem to ZFS and is a pretty uncommon use-case anyway. Also you cannot grow a ZFS "raidz" vdev by adding disks - but you cannot do that with any typical RAID5/6 array either. However if you want to add disks to a raidz then you can workaround the RAID5/6 striped parity problem by adding the additional disks into a new vdev and adding that new vdev into the existing ZFS volume (named "tank"). The kind of storage arrays that grow by adding solitary disks work quite different to your typical RAID. They'll often have designated parity drives instead. See solutions like "unraid". > LVM and mdadm manage these jobs much better and are composable to your needs. LVM and mdadm are not file systems though, so they only solve part of the problem. The fact ZFS handles volume management, software RAIDing, file system operations (and a bunch of other stuff that your typical fs doesn't yet support (eg snapshotting)) and does so from surprisingly intuitive command line tools is what edges ZFS for me in terms of usability. Ultimately though, "easier" is such a broad term that we're essentially just arguing personal opinion. :)
- zaarn 8y ago
  >start making assumptions about it's performance on low memory systems. I've ran it for real. Once a few more background services start running, ZFS bogs down and keels over. I've had to reboot the NAS dozens of times because the entire filesystem had become totally unresponsive. >You actually can grow a ZFS vdev - exactly by replacing one disk at a time (albeit you don't get the gained capacity until all the disks in the vdev have been replaced). Which is not an accaptable answer to the situation I presented, you still have to buy multiple disks to grow the pool, you can't do it one at a time. >The kind of storage arrays that grow by adding solitary disks work quite different to your typical RAID. They'll often have designated parity drives instead. See solutions like "unraid". Not only unraid, there are other systems that provide almost this. Syno and QNAP can both grow an array in storage capacity if you add a single new disk, even if it's larger and once you have added enough to provide parity, the additional capacity will become used too. >surprisingly intuitive command line tools is what edges ZFS for me in terms of usability. I found the commands anything but intuitive though I guess that's a difference of a few years of experience, I'd agree that would be personal opinion if it's easier.
  - laumars 8y ago
    > I've ran it for real. Once a few more background services start running, ZFS bogs down and keels over. I've had to reboot the NAS dozens of times because the entire filesystem had become totally unresponsive. Was that an off the shelf NAS? If so then I'm guessing it wasn't exactly high spec'ed even putting aside the RAM? eg did it have a 32bit CPU? I've heard that does a massive amount to dent ZFS's performance (I wouldn't know because I've only ever ran on it 64 bit systems). As I said, I've not had any issues with raidz2 on 2GB RAM. I've even ran virtual machines on one OpenSolaris box with ZFS and 2GB RAM so these weren't always dedicated storage servers either. edit: just read elsewhere you were running FreeNAS on a Pentium 4. Yeah that would do it. FreeNAS wasn't optimised for low end systems (in fact they outright tell you not to!) and the P4 is pretty low end CPU as well. I still think ZFS would have ran on that but you'd have been better off going with vanilla FreeBSD instead. > Which is not an accaptable answer to the situation I presented, you still have to buy multiple disks to grow the pool, you can't do it one at a time. ...but you can't do that with most traditional RAIDs either and the few you can do it on require you to take the whole storage pool offline while you grow the pool then resize the file system. That process can literally take several days on some storage servers. So ZFS might require you to add vdevs to the pool (tank) but at least it can all be done on live systems without any downtime. That in itself is a massive boon for ease of use in my opinion. To be honest, it's pretty rare you want to upgrade a storage array by just one disk anyway. At least outside of home users anyway. And if you are the type of person that would need to do that then you wouldn't go for a traditional raid5/6 style pool either. Thus the argument becomes a comparison of apples and oranges as you'e comparing completely different types of software RAIDing. > Not only unraid I didn't say "exclusively unraid". I said "solutions like unraid". I'm fully aware there are other solutions out there. > there are other systems that provide almost this. Syno and QNAP can both grow an array in storage capacity if you add a single new disk, even if it's larger and once you have added enough to provide parity, the additional capacity will become used too. When you said "there are other systems that provide almost this" and "once you have added enough [disks] to provide parity, the additional capacity will become used too." I really had to question what your actual point of this discussion was because just a paragraph earlier you were calling my points "unacceptable" because ZFS required you to add more than 1 extra disk to provide parity; and now you're saying your preferred solutions also does the same? I hope I'm just being presumptuous here but it's starting to feel like you're just arguing for the sake of arguing. :-/ > I found the commands anything but intuitive though I guess that's a difference of a few years of experience Maybe. I've been a Linux and UNIX sysadmin for a fair few years now so experience is not something I'm short of. :) Honest question: what is your experience of ZFS outside of the troubles you had on your NAS? I could totally see why the problems you described would taint your opinion of ZFS but I definitely wouldn't say what you experienced is typical for ZFS. In any case, I'm not going to argue that everyone should be running ZFS - it's just technology, you can run whatever you want to run.
NeedMoreTea 8y ago
> doesn't run well with 1-2GB regardless of how many disks you have By default, sure. There is a FreeBSD (I think) guide to tuning ZFS that lets you tune it for low memory. It's a long time since I've done this so I can't remember which settings, but I ran an old machine tuned for its low max memory, with 4 drives in RaidZ, and it was just fine. The FreeNAS forum doesn't help much with this as the default opinion there seems to be 16GB ECC. :)

by zaarn 8y ago

laumars 8y ago

> doesn't run well with 1-2GB regardless of how many disks you have I've read people say that but I've never had any issues when I used to run ZFS on various systems with 2GB RAM. So I wonder how many of those people actually had practical experiences running ZFS on low memory systems and how much of it is people just looking at some of the recommended memory specifications (which are high) and start making assumptions about it's performance on low memory systems. > RAID1 mirrors, leaving consumers unable to grow pools 1 disk at a time because ZFS is fundamentally unable to grow a RAID vdev You actually can grow a ZFS vdev - exactly by replacing one disk at a time (albeit you don't get the gained capacity until all the disks in the vdev have been replaced). What you cannot do is shrink it - but that's not a unique problem to ZFS and is a pretty uncommon use-case anyway. Also you cannot grow a ZFS "raidz" vdev by adding disks - but you cannot do that with any typical RAID5/6 array either. However if you want to add disks to a raidz then you can workaround the RAID5/6 striped parity problem by adding the additional disks into a new vdev and adding that new vdev into the existing ZFS volume (named "tank"). The kind of storage arrays that grow by adding solitary disks work quite different to your typical RAID. They'll often have designated parity drives instead. See solutions like "unraid". > LVM and mdadm manage these jobs much better and are composable to your needs. LVM and mdadm are not file systems though, so they only solve part of the problem. The fact ZFS handles volume management, software RAIDing, file system operations (and a bunch of other stuff that your typical fs doesn't yet support (eg snapshotting)) and does so from surprisingly intuitive command line tools is what edges ZFS for me in terms of usability. Ultimately though, "easier" is such a broad term that we're essentially just arguing personal opinion. :)

zaarn 8y ago
>start making assumptions about it's performance on low memory systems. I've ran it for real. Once a few more background services start running, ZFS bogs down and keels over. I've had to reboot the NAS dozens of times because the entire filesystem had become totally unresponsive. >You actually can grow a ZFS vdev - exactly by replacing one disk at a time (albeit you don't get the gained capacity until all the disks in the vdev have been replaced). Which is not an accaptable answer to the situation I presented, you still have to buy multiple disks to grow the pool, you can't do it one at a time. >The kind of storage arrays that grow by adding solitary disks work quite different to your typical RAID. They'll often have designated parity drives instead. See solutions like "unraid". Not only unraid, there are other systems that provide almost this. Syno and QNAP can both grow an array in storage capacity if you add a single new disk, even if it's larger and once you have added enough to provide parity, the additional capacity will become used too. >surprisingly intuitive command line tools is what edges ZFS for me in terms of usability. I found the commands anything but intuitive though I guess that's a difference of a few years of experience, I'd agree that would be personal opinion if it's easier.
- laumars 8y ago
  > I've ran it for real. Once a few more background services start running, ZFS bogs down and keels over. I've had to reboot the NAS dozens of times because the entire filesystem had become totally unresponsive. Was that an off the shelf NAS? If so then I'm guessing it wasn't exactly high spec'ed even putting aside the RAM? eg did it have a 32bit CPU? I've heard that does a massive amount to dent ZFS's performance (I wouldn't know because I've only ever ran on it 64 bit systems). As I said, I've not had any issues with raidz2 on 2GB RAM. I've even ran virtual machines on one OpenSolaris box with ZFS and 2GB RAM so these weren't always dedicated storage servers either. edit: just read elsewhere you were running FreeNAS on a Pentium 4. Yeah that would do it. FreeNAS wasn't optimised for low end systems (in fact they outright tell you not to!) and the P4 is pretty low end CPU as well. I still think ZFS would have ran on that but you'd have been better off going with vanilla FreeBSD instead. > Which is not an accaptable answer to the situation I presented, you still have to buy multiple disks to grow the pool, you can't do it one at a time. ...but you can't do that with most traditional RAIDs either and the few you can do it on require you to take the whole storage pool offline while you grow the pool then resize the file system. That process can literally take several days on some storage servers. So ZFS might require you to add vdevs to the pool (tank) but at least it can all be done on live systems without any downtime. That in itself is a massive boon for ease of use in my opinion. To be honest, it's pretty rare you want to upgrade a storage array by just one disk anyway. At least outside of home users anyway. And if you are the type of person that would need to do that then you wouldn't go for a traditional raid5/6 style pool either. Thus the argument becomes a comparison of apples and oranges as you'e comparing completely different types of software RAIDing. > Not only unraid I didn't say "exclusively unraid". I said "solutions like unraid". I'm fully aware there are other solutions out there. > there are other systems that provide almost this. Syno and QNAP can both grow an array in storage capacity if you add a single new disk, even if it's larger and once you have added enough to provide parity, the additional capacity will become used too. When you said "there are other systems that provide almost this" and "once you have added enough [disks] to provide parity, the additional capacity will become used too." I really had to question what your actual point of this discussion was because just a paragraph earlier you were calling my points "unacceptable" because ZFS required you to add more than 1 extra disk to provide parity; and now you're saying your preferred solutions also does the same? I hope I'm just being presumptuous here but it's starting to feel like you're just arguing for the sake of arguing. :-/ > I found the commands anything but intuitive though I guess that's a difference of a few years of experience Maybe. I've been a Linux and UNIX sysadmin for a fair few years now so experience is not something I'm short of. :) Honest question: what is your experience of ZFS outside of the troubles you had on your NAS? I could totally see why the problems you described would taint your opinion of ZFS but I definitely wouldn't say what you experienced is typical for ZFS. In any case, I'm not going to argue that everyone should be running ZFS - it's just technology, you can run whatever you want to run.

NeedMoreTea 8y ago

> doesn't run well with 1-2GB regardless of how many disks you have By default, sure. There is a FreeBSD (I think) guide to tuning ZFS that lets you tune it for low memory. It's a long time since I've done this so I can't remember which settings, but I ran an old machine tuned for its low max memory, with 4 drives in RaidZ, and it was just fine. The FreeNAS forum doesn't help much with this as the default opinion there seems to be 16GB ECC. :)