4 ms·

A lot of really nice, focused polish items in this release that have real practical benefits. The corrective "healing" receive for example, ZFS has always been

by xoa 3y ago

A lot of really nice, focused polish items in this release that have real practical benefits. The corrective "healing" receive for example, ZFS has always been able self-heal when there is redundancy (either device level or using the copies attribute), and that's always been a core feature. And in its original incarnation there'd always be at least some redundancy. But as it gets wider use fact is the vast majority of systems have a single device and don't want to cut available storage in half for copies=1. Corruption can still be detected with a scrub thanks to checksums, so ZFS is still useful anyway. It's better to know about corruption asap, while you still have good backups. But being able to smoothly and very quickly repair that without any local redundancy using backups you should have anyway is a nice bit of polish and leveraging of the foundation. One question if anyone who follows it more closely knows: are there any efforts to work specifically on ZVOL performance? They're very handy for iSCSI and other features, but my understanding is they haven't gotten much focus for quite awhile now. Maybe that just doesn't have any real company backing right now for R&D but I hope it gets some attention eventually.

brnt 3y ago
> But as it gets wider use fact is the vast majority of systems have a single device and don't want to cut available storage in half for copies=1 I'm still patiently holding out for the first (current, usable) filesystem that let's us set a parity fraction and let me use FEC to heal small errors.
- xoa 3y ago
  I suppose in principle you could partition a drive n ways the run RAIDZ1? Though performance would probably be mediocre and there'd be some oddities. Occurs to me know that as a practical matter perhaps raw storage will solve this for a lot of people. The pace of increase in storage/$ still seems to be steady, and in turn continues to mean more and more people trivially have their total needs exceeded. At some point may be perfectly reasonable to throw half at redundancy?
  - GrilledChips 3y ago
    This is a bad idea. If you want repair, you need to either set more than one copy in ZFS (still pretty dumb) or use multiple drives in a raidz (or mirror, though raidz is more efficient). When you try to partition a drive like this (or configure ZFS to write multiple copies) you loose everything in the event that the whole drive fails. Honestly, it's a far better plan to just use multiple drives in a raidz so that whole-drive failures can be reconstructed in addition to data corruption.
    - xoa 3y ago
      Please read the thread before commenting. The scenarios we're discussing are explicitly in the context of single storage device systems getting helped by the new healing replication feature. Huge numbers of systems do not have multiple drives nor even necessarily the ability to have multiple drives. Nobody is suggesting it's optimal. I've run ZFS for 12+ years now with hundreds of systems. You don't need to tell us the obvious.
- GrilledChips 3y ago
  Use raidz.
- throw0101c 3y ago
  A one-drive Ceph ECC pool? * https://docs.ceph.com/en/latest/rados/operations/erasure-code/ https://docs.ceph.com/en/latest/rados/operations/erasure-cod...
aeadio 3y ago
> But as it gets wider use fact is the vast majority of systems have a single device and don't want to cut available storage in half for copies=1. Setting copies=2 for the root filesystem, but segmenting stuff like caches, user downloads, etc into separate datasets with copies=1, is totally reasonable for this purpose. The base system in most distributions only takes a few GiB. 1 TiB SSDs are pretty common even on entry level laptops these days, and it's becoming increasingly difficult to find new machines without at least 512 GiB.
GrilledChips 3y ago
A scrub can both detect and repair damage. You don't need to set additional copies. If you don't want to buy 4TB of drives to store 2TB of data in a mirror, use a raidz instead. In a raidz with at least one parity disk (and any number of data disks), corruption found on any data disk can be repaired immediately. For non-overlapping corruption a single parity disk is enough. If you have corruption of the same block on two different disks (extremely unlikely) then this can be repaired if you have two parity disks. (Etc. for three identically-corrupt data disks and three parity disks) You can have eg. two parity disks and eight data disks (or more) and so long as no more than two disks are corrupted in the same block at the same time then a scrub will repair it completely. The system of parity disks in ZFS' raidz is meant for reconstructing entire failed data disks, so fixing corruption is no big deal.
magicalhippo 3y ago
> are there any efforts to work specifically on ZVOL performance There's this[1] work which integrated ZFS better with the kernel so it could merge small IOs on zvols better. It was merged almost a year ago. However due to a recent issue[2] it has been disabled until they can figure out what exactly is going on. If they can get it working it seemed to give a decent boost for certain scenarios. [1]: https://github.com/openzfs/zfs/pull/13148 https://github.com/openzfs/zfs/pull/13148 [2]: https://github.com/openzfs/zfs/issues/15351 https://github.com/openzfs/zfs/issues/15351

by xoa 3y ago

brnt 3y ago

> But as it gets wider use fact is the vast majority of systems have a single device and don't want to cut available storage in half for copies=1 I'm still patiently holding out for the first (current, usable) filesystem that let's us set a parity fraction and let me use FEC to heal small errors.

xoa 3y ago
I suppose in principle you could partition a drive n ways the run RAIDZ1? Though performance would probably be mediocre and there'd be some oddities. Occurs to me know that as a practical matter perhaps raw storage will solve this for a lot of people. The pace of increase in storage/$ still seems to be steady, and in turn continues to mean more and more people trivially have their total needs exceeded. At some point may be perfectly reasonable to throw half at redundancy?
- GrilledChips 3y ago
  This is a bad idea. If you want repair, you need to either set more than one copy in ZFS (still pretty dumb) or use multiple drives in a raidz (or mirror, though raidz is more efficient). When you try to partition a drive like this (or configure ZFS to write multiple copies) you loose everything in the event that the whole drive fails. Honestly, it's a far better plan to just use multiple drives in a raidz so that whole-drive failures can be reconstructed in addition to data corruption.
  - xoa 3y ago
    Please read the thread before commenting. The scenarios we're discussing are explicitly in the context of single storage device systems getting helped by the new healing replication feature. Huge numbers of systems do not have multiple drives nor even necessarily the ability to have multiple drives. Nobody is suggesting it's optimal. I've run ZFS for 12+ years now with hundreds of systems. You don't need to tell us the obvious.
GrilledChips 3y ago
Use raidz.
throw0101c 3y ago
A one-drive Ceph ECC pool? * https://docs.ceph.com/en/latest/rados/operations/erasure-code/ https://docs.ceph.com/en/latest/rados/operations/erasure-cod...

aeadio 3y ago

> But as it gets wider use fact is the vast majority of systems have a single device and don't want to cut available storage in half for copies=1. Setting copies=2 for the root filesystem, but segmenting stuff like caches, user downloads, etc into separate datasets with copies=1, is totally reasonable for this purpose. The base system in most distributions only takes a few GiB. 1 TiB SSDs are pretty common even on entry level laptops these days, and it's becoming increasingly difficult to find new machines without at least 512 GiB.

GrilledChips 3y ago

A scrub can both detect and repair damage. You don't need to set additional copies. If you don't want to buy 4TB of drives to store 2TB of data in a mirror, use a raidz instead. In a raidz with at least one parity disk (and any number of data disks), corruption found on any data disk can be repaired immediately. For non-overlapping corruption a single parity disk is enough. If you have corruption of the same block on two different disks (extremely unlikely) then this can be repaired if you have two parity disks. (Etc. for three identically-corrupt data disks and three parity disks) You can have eg. two parity disks and eight data disks (or more) and so long as no more than two disks are corrupted in the same block at the same time then a scrub will repair it completely. The system of parity disks in ZFS' raidz is meant for reconstructing entire failed data disks, so fixing corruption is no big deal.

magicalhippo 3y ago

> are there any efforts to work specifically on ZVOL performance There's this[1] work which integrated ZFS better with the kernel so it could merge small IOs on zvols better. It was merged almost a year ago. However due to a recent issue[2] it has been disabled until they can figure out what exactly is going on. If they can get it working it seemed to give a decent boost for certain scenarios. [1]: https://github.com/openzfs/zfs/pull/13148 https://github.com/openzfs/zfs/pull/13148 [2]: https://github.com/openzfs/zfs/issues/15351 https://github.com/openzfs/zfs/issues/15351