5 ms·

I noticed that the author is using ZFS-native encryption, which in my experience is not particularly stable. I've even managed to corrupt an entire pool with ZF

I noticed that the author is using ZFS-native encryption, which in my experience is not particularly stable. I've even managed to corrupt an entire pool with ZFS send/receives when trying to make backups. I'd strongly recommend using ZFS-on-LUKS instead if encryption is required. The list of open issues on GitHub in the native encryption component is quite telling: https://github.com/openzfs/zfs/issues?q=is%3Aopen+is%3Aissue+label%3A%22Component%3A+Encryption%22 https://github.com/openzfs/zfs/issues?q=is%3Aopen+is%3Aissue...

Modified3019 3y ago
I have to agree, the leadership’s response and handling of errors in regards to encryption has been… honestly pretty disappointing to see as a zfs fan and user. While it’s not common, the amount of people running into edge cases and killing both their sending and sometimes even receiving pools (better have another backup!) is frankly unacceptable. “Raw sends” seem to be especially at risk, though sending in general seems to be where the issues mostly lay. My thoughts mirror the comments here: https://discourse.practicalzfs.com/t/is-native-encryption-ready-for-production-use/532/2 https://discourse.practicalzfs.com/t/is-native-encryption-re... Here’s a currently unmaintained document by Rincebrain that he was using to try to track things before he got burned out by the lack or response. https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg/htmlview https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6sww...
aborsy 3y ago
I use it with snapshots and ZFS send, and so far it’s been fine. Truenas users also may use it. But I’m concerned reading these comments. Anyone else experiencing issues?
- Fnoord 3y ago
  I've been using ZFS with native encryption (Ubuntu Server) but also ZFS with LUKS (Arch, Ubuntu Desktop). Zero issues (though inability to run latest kernel can be annoying, esp on rolling distro). Wouldn't surprise me if write cache has a role in this issue though.
- secabeen 3y ago
  The fundamental issue with ZFS encryption is that the primary developer that created it is no longer contributing significantly to the project. It's good code, with good tests, but it's not getting any additional love. The utilities and tooling surrounding encryption are also weak, and there are ways you can throw away critical, invisible keydata without realizing it, and no tool to allow the correction of the issue, even if you have the missing keydata on another system.
  - aborsy 3y ago
    It’s interesting that there hasn’t been anyone since to continue developing Tom’s code. I wonder how long the situation might continue. If no one wants to take over in near future, they might have to remove the feature.
didntcheck 3y ago
I was quite surprised when I learned that ZFS-native encryption is so underdeveloped, and that the developers still sometimes seem to forget about it when coding new features. I had presumed it would be table stakes for enterprise storage by now. Is it just the case that everyone uses it on GELI or LUKS, or am I wrong to presume that encryption is as widespread in enterprise deployments as I thought? It's a shame, because the main draw of the native encryption to me is being able to have a zero-trust backup target with all the advantages of zfs send over user-level backup software. But I've heard of several people running into issues doing this (though luckily no actual data loss that I've heard, just headaches)
- sneak 3y ago
  I think large enterprises usually do encryption on the application level and not storage level these days. A KMS or encryption service, then storage services just hold encrypted blobs. I run the 50 zfs disks in my house on LUKS.

Modified3019 3y ago

I have to agree, the leadership’s response and handling of errors in regards to encryption has been… honestly pretty disappointing to see as a zfs fan and user. While it’s not common, the amount of people running into edge cases and killing both their sending and sometimes even receiving pools (better have another backup!) is frankly unacceptable. “Raw sends” seem to be especially at risk, though sending in general seems to be where the issues mostly lay. My thoughts mirror the comments here: https://discourse.practicalzfs.com/t/is-native-encryption-ready-for-production-use/532/2 https://discourse.practicalzfs.com/t/is-native-encryption-re... Here’s a currently unmaintained document by Rincebrain that he was using to try to track things before he got burned out by the lack or response. https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6swwBZXgXwdCPKgp4SbPZwTexCg/htmlview https://docs.google.com/spreadsheets/d/1OfRSXibZ2nIE9DGK6sww...

aborsy 3y ago

I use it with snapshots and ZFS send, and so far it’s been fine. Truenas users also may use it. But I’m concerned reading these comments. Anyone else experiencing issues?

Fnoord 3y ago
I've been using ZFS with native encryption (Ubuntu Server) but also ZFS with LUKS (Arch, Ubuntu Desktop). Zero issues (though inability to run latest kernel can be annoying, esp on rolling distro). Wouldn't surprise me if write cache has a role in this issue though.
secabeen 3y ago
The fundamental issue with ZFS encryption is that the primary developer that created it is no longer contributing significantly to the project. It's good code, with good tests, but it's not getting any additional love. The utilities and tooling surrounding encryption are also weak, and there are ways you can throw away critical, invisible keydata without realizing it, and no tool to allow the correction of the issue, even if you have the missing keydata on another system.
- aborsy 3y ago
  It’s interesting that there hasn’t been anyone since to continue developing Tom’s code. I wonder how long the situation might continue. If no one wants to take over in near future, they might have to remove the feature.

didntcheck 3y ago

I was quite surprised when I learned that ZFS-native encryption is so underdeveloped, and that the developers still sometimes seem to forget about it when coding new features. I had presumed it would be table stakes for enterprise storage by now. Is it just the case that everyone uses it on GELI or LUKS, or am I wrong to presume that encryption is as widespread in enterprise deployments as I thought? It's a shame, because the main draw of the native encryption to me is being able to have a zero-trust backup target with all the advantages of zfs send over user-level backup software. But I've heard of several people running into issues doing this (though luckily no actual data loss that I've heard, just headaches)

sneak 3y ago
I think large enterprises usually do encryption on the application level and not storage level these days. A KMS or encryption service, then storage services just hold encrypted blobs. I run the 50 zfs disks in my house on LUKS.