5 ms·

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meani

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. Other fun things are the flexible block sizes and their relation to the size you're writing and compression ... Chris Siebenmann has written quite a bit about it (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSLogicalVsPhysicalBlockSizes https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSLogicalV...). One thing I'm particularly interested in is to see if this new patch offers a way to decrease fragmentation on existing and loaded pools (allocation changes if they are too full, and this patch will for the first time allow us to avoid building a completely new pool). [edit] The PR is here: https://github.com/openzfs/zfs/pull/12225 https://github.com/openzfs/zfs/pull/12225 I also recommend reading the discussions in the ZFS repository - they are quite interesting and reveal a lot of the reasoning behind the filesystem. Recommended even to people who don't write filesystems as a living.

chungy 5y ago
> The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. This is generally in-line with other ZFS operations. For example, changing compression policies will not rewrite existing data and only new data is affected. It simplifies some code paths and keeps performance good no matter what. You don't get a surprising reduction on performance.
- deleted 5y ago
  [deleted]
GauntletWizard 5y ago
I'm immensely pleased with the approach they've chosen, because it offers the possibility of improving your stripe size, without requiring commitment to. New blocks are written with the new stripe size, so all you have to do to improve your stripe width is rewrite all of your files (and deal with the snapshot churn that will cause). It also offers the tempting possibility of the reverse, where you can create new files with stripes and then rewrite all your data to reduce capacity. I used to work at Isilon, and the design of this feature looks very familiar to me: That's a good thing. I'm excited for the possibilities for ZFS going forward. I hope they'll come up with a restriper tool that will allow you to rewrite all of your files with the new block layout without breaking snapshots, etc, but either way, this is a huge advancement for my own personal use of ZFS.
paulmd 5y ago
There is also a linear degradation of metadata/blockinfo performance (in the order of the number of rewrites) due to the need to check multiple possible locations for the block pointer. In practice this probably just means more RAM utilization but... eh. Even as a home user, I've become convinced the juice is not worth the squeeze. For serious use, just buy four drives at a time and upgrade four drives at a time. That gives you 4-drive RAIDZ1 in one VDEV and if you want you can expand to a second VDEV or upgrade the capacity of the drives in your existing VDEV. If you don't want to do four drives at once, you can still use ZFS with 1 drive = 1 pool (or 2 mirrored drives = 1 pool for redundancy) and just manually distribute files over the volumes. No, it won't automatically do it like Unraid but perhaps you can layer unraid over the top somehow if you really insist (otherwise there's also solutions like git-annex). It still gets you far better data integrity guarantees and far better reliability than any available alternatives. Maybe it is just stockholm syndrome but do you really, really need to dynamically expand your array by an arbitrary number of disks that badly? There is ultimately no way around the reliability question so you can't expand forever and if you're serious about setting up a nice fileserver that'll be reliable for the long run, putting 4 drives in it isn't that bad. And ZFS is no worse than the alternatives on 1-drive or 2-drive scenarios in terms of reliability/expansion. Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. There is also the possibility of manually doing some weird shit like putting 2 partitions on each disk so you could move one member of the VDEV over to another disk if you expand, if you start with 2 disks with 2 partitions each then you could grow to 4 disks without issue, which would lower the increment even further.
- WastingMyTime89 5y ago
  > Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. No, it's not. Btrfs is nowadays perfectly reliable as long as you avoid the in development features which are all explicitly marked as in development and will warn you. That's why Facebook uses it in production. It also has some nice advantage for home use. Btrfs pools are a lot simpler to manage and have less quirks than ZFS. > Maybe it is just stockholm syndrome but do you really, really need to dynamically expand your array by an arbitrary number of disks that badly? There is ultimately no way around the reliability question so you can't expand forever and if you're serious about setting up a nice fileserver that'll be reliable for the long run, putting 4 drives in it isn't that bad. Btrfs will do that just fine for example. That's really a limitation of ZFS. I wish ZFS users would just stop FUDing about btrfs. Most of them haven't touched it for a decade and keep paroting the same old things.

chungy 5y ago

> The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. This is generally in-line with other ZFS operations. For example, changing compression policies will not rewrite existing data and only new data is affected. It simplifies some code paths and keeps performance good no matter what. You don't get a surprising reduction on performance.

deleted 5y ago
[deleted]

GauntletWizard 5y ago

I'm immensely pleased with the approach they've chosen, because it offers the possibility of improving your stripe size, without requiring commitment to. New blocks are written with the new stripe size, so all you have to do to improve your stripe width is rewrite all of your files (and deal with the snapshot churn that will cause). It also offers the tempting possibility of the reverse, where you can create new files with stripes and then rewrite all your data to reduce capacity. I used to work at Isilon, and the design of this feature looks very familiar to me: That's a good thing. I'm excited for the possibilities for ZFS going forward. I hope they'll come up with a restriper tool that will allow you to rewrite all of your files with the new block layout without breaking snapshots, etc, but either way, this is a huge advancement for my own personal use of ZFS.

paulmd 5y ago

There is also a linear degradation of metadata/blockinfo performance (in the order of the number of rewrites) due to the need to check multiple possible locations for the block pointer. In practice this probably just means more RAM utilization but... eh. Even as a home user, I've become convinced the juice is not worth the squeeze. For serious use, just buy four drives at a time and upgrade four drives at a time. That gives you 4-drive RAIDZ1 in one VDEV and if you want you can expand to a second VDEV or upgrade the capacity of the drives in your existing VDEV. If you don't want to do four drives at once, you can still use ZFS with 1 drive = 1 pool (or 2 mirrored drives = 1 pool for redundancy) and just manually distribute files over the volumes. No, it won't automatically do it like Unraid but perhaps you can layer unraid over the top somehow if you really insist (otherwise there's also solutions like git-annex). It still gets you far better data integrity guarantees and far better reliability than any available alternatives. Maybe it is just stockholm syndrome but do you really, really need to dynamically expand your array by an arbitrary number of disks that badly? There is ultimately no way around the reliability question so you can't expand forever and if you're serious about setting up a nice fileserver that'll be reliable for the long run, putting 4 drives in it isn't that bad. And ZFS is no worse than the alternatives on 1-drive or 2-drive scenarios in terms of reliability/expansion. Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. There is also the possibility of manually doing some weird shit like putting 2 partitions on each disk so you could move one member of the VDEV over to another disk if you expand, if you start with 2 disks with 2 partitions each then you could grow to 4 disks without issue, which would lower the increment even further.

WastingMyTime89 5y ago
> Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. No, it's not. Btrfs is nowadays perfectly reliable as long as you avoid the in development features which are all explicitly marked as in development and will warn you. That's why Facebook uses it in production. It also has some nice advantage for home use. Btrfs pools are a lot simpler to manage and have less quirks than ZFS. > Maybe it is just stockholm syndrome but do you really, really need to dynamically expand your array by an arbitrary number of disks that badly? There is ultimately no way around the reliability question so you can't expand forever and if you're serious about setting up a nice fileserver that'll be reliable for the long run, putting 4 drives in it isn't that bad. Btrfs will do that just fine for example. That's really a limitation of ZFS. I wish ZFS users would just stop FUDing about btrfs. Most of them haven't touched it for a decade and keep paroting the same old things.