9 ms·
Playing with ZFS encryption on Linux
- symlinkk 9y agonice post, I think you forgot to fill in the Introduction though
- funkaster 9y agothanks! I think I stupidly lost that paragraph in a merge. I'll remove it for now and add it later. Thanks again!
- ubercow 9y agoI'm not really familiar with the development process between ZFS on Linux and ZFS on BSD, do these pull requests usually get merged upstream for use on BSD, Solaris, etc?
- aidenn0 9y agoOpenZFS[1] is upstream for all of the currently maintained open-source ZFS implementations, including ZoL and FreeBSD. 1: http://open-zfs.org/wiki/Main_Page http://open-zfs.org/wiki/Main_Page
- boomboomsubban 9y agoSolaris is completely separate, but I believe the encryption comes from Illumos just needs a ton of changing to work on Linux.
- binwiederhier 9y agoJust the crypto primitives are from Illinois iirc, the actual scheme is fairly new.
- dom0 9y agoDidn't Oracle ZFS have encryption for some time now? (Was that after closing Solaris?)
- lathiat 9y agoyeah it was done after closing this is an independent implementation for the open source side
- X86BSD 9y agoYes and they have a poor implementation for encryption at that. Pawel who did the port of zfs to FreeBSD did not like how they implemented it. As far as zfs goes oracle will be the odd man out compatibility wise with the rest of the platforms supporting zfs and the openzfs encryption scheme.
- tcaputi 9y agoOracle ZFS does have encryption, but this implementation fixes a few security / usability issues with it, adds more features, and brings the implementation into the open source world.
- AndrewDavis 9y agoTogether FreeBSD, ZoL and the Illumos groups work on OpenZFS. Oracle closed off Solaris. The decendent of Open Solaris is Illumos. Oracle ZFS and OpenZFS are not hte same thing. OpenZFS repo is the upstream, though it is closely tracks the Illumos version. From there the projects pull on OpenZFS to create their implementations. New features are developed independently by the projects with rules that a new feature must be enabled in a downstream distro for X period of time before it can be upstreamed into OpenZFS.
- ryao 9y agoLet's not forget Mac OS X and OSv, although the amount of work being done that and shared is not as large as it is with the other 3 platforms.
- aidenn0 9y ago64 bytes seems like an arbitrarily short limit for passphrases; it's going to be hashed at some point anyway (preferably with a decent KDF designed for the purpose) and 64 bytes is nowhere near long enough for natural English language text to be used to derive 32 bytes of entropy (estimates of English language text are ~1.5bits of entropy per character).
- RJIb8RBYxzAMX9u 9y agoAt the risk of invoking Cunningham's Law, nobody uses passphrases anywhere near 64 bytes that he / she has to memorize. And if you're using a password manager, just use the raw or hex key option. 32 bytes => 64 hex digits, which is probably not a coincidence. :-)
- takeda 9y agoPassword yes, passphprase it's quite possible.
- takeda 9y agoEh, Materialistic doesn't allow me to edit my comment. I mean you're right about passwords, but a passphrases (e.g. a sentences) are easy to remember, more secure and can often be longer than 64 bytes.
- yjftsjthsd-h 9y agoEr, of course it's not a coincidence; hex is base 16, so it takes 4 bits to represent each digit, which is half a byte. That is, hex always is 2 digits per byte. Or did I miss your point?
- Freaky 9y ago> At the risk of invoking Cunningham's Law, nobody uses passphrases anywhere near 64 bytes that he / she has to memorize. http://52.24.230.241/bc/password_generation.php http://52.24.230.241/bc/password_generation.php This produces memorable passphrases pretty close to that limit, as described in http://www-scf.usc.edu/~mghazvin/papers/marjan15.pdf http://www-scf.usc.edu/~mghazvin/papers/marjan15.pdf
- binwiederhier 9y agoA little more details on the crypto aspects: https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/ https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zf... (This is my post. I hope it's okay to post this here.)
- funkaster 9y agoThis is really good! I was trying to find some time to write about the crypto details of it, but your post is a great summary :) I'll add a reference to it in my post.
- heftysync 9y agoHave the design decisions been reviewed by a cryptographer yet? According to the presentation on youtube, it had not.
- ryao 9y agoWe are having trouble finding a volunteer.
- cvwright 9y agoI'm not a cryptographer, but for your dedup version, you might want to consider using SIV mode [1,2]. It provides a well known, already vetted deterministic authenticated encryption, so you don't have to build your own from HMAC etc and then get it vetted. [1] http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/siv/siv.pdf http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/propose... [2] https://tools.ietf.org/html/rfc5297 https://tools.ietf.org/html/rfc5297
- tcaputi 9y agoThank you for the suggestion. As I said above, there is nothing preventing something like that from being implemented in ZFS in the future. Unfortunately, however, the encryption implementation uses a port of the Illumos Kernel Crypto Framework, which has not yet implemented an SIV mode. As far as using our own HMAC goes we are using a standard SHA512-HMAC implementation which should be just as secure.
- eslaught 9y agoDoes anyone know a good guide for getting set up with ZFS (particularly on Linux/Ubuntu)? The Ubuntu wiki basically just say "apt install zfs" and then links to a page that appears to be a brain dump of all the possible commands. Something with more coherence explanation of the relevant concepts would be really helpful.
- rincebrain 9y ago[1] is a nice explanation of getting set up with ZFS root on Ubuntu 16.04 with explanations for what each of the commands are doing. I'm not in a good position to judge whether this is the correct level of detail for someone new to ZFS; it seems slightly lighter on explanation of concepts than I'd prefer. [2] seems reasonably nice at a quick look. [1] - https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-... [2] - http://kbdone.com/zfs-basics/ http://kbdone.com/zfs-basics/
- funkaster 9y agoif it's about management, then you might want to look at the docs I recommend in the post: Oracle's docs are very detailed[1] and the freebsd handbook[2] is very useful as well. [1]: http://docs.oracle.com/cd/E19253-01/819-5461/gamnq/index.html http://docs.oracle.com/cd/E19253-01/819-5461/gamnq/index.htm... [2]: https://www.freebsd.org/doc/handbook/zfs.html https://www.freebsd.org/doc/handbook/zfs.html
- sargun 9y agoWhat are you setting up ZFS for?
- weitzj 9y agoI have documented the steps I had to take here https://janweitz.de/article/creating-a-zfs-zroot-raid-10-on-ubuntu-16.04/ https://janweitz.de/article/creating-a-zfs-zroot-raid-10-on-... Be aware: Uefi did not work for me in VirtualBox with ZFS. Booting with UEFI, ZFS and RAIDz works, but I did not figure out how to install the UEFI grub-boot on all RAIDz drives. Therefore, if the one and only drive with the boot partition fails, I have to rely on an USB-stick as a backup-plan for the bootloader. Probably if I could start over I would try to skip UEFI and use legacy booting and installing Grub in the MBR of each drive (if that is possible). For encryption we are using Luks right now, since it encrypts block devices and ZFS uses these block devices to form its raid. Luks needs to decrypt ALL raid devices inside the Grub boot prompt in order for ZFS to start its raidz. To make this happen, we had to modify: /usr/share/initramfs-tools/hooks/cryptroot and remove an early return in a for-loop in get_fs_devices() , since otherwise only the first device was decrypted. Make sure /etc/lvm/lvm.conf contains use_lvmetad = 0 Make sure /etc/default/grub contains GRUB_ENABLE_CRYPTODISK=y GRUB_CMDLINE_LINUX_DEFAULT=“text" So, yeah, I am looking forward for zfs-native encryption on Linux to make booting "safer/easier".
- XorNot 9y agoI've been really looking forward to OpenZFS encryption support. ZFS on LUKS leaves a lot to be desired, and I've only just recently begun trying to shift all my PCs and devices to be fully encrypted by default.
- FullyFunctional 9y agoWill this work the same way on FreeBSD/FreeNAS when this eventually lands? OT: one of the sed lines is hard to read sed -i 's/github.com\/zfsonlinux\/zfs.git/github.com\/tcaputi\/zfs.git/' PKGBUILD the OP might leverage that you can use other quoting characters, like sed -i 's,github.com/zfsonlinux/zfs.git,github.com/tcaputi/zfs.git,' PKGBUILD
- Amezarak 9y agoFreeNAS has supported zfs encryption for a long while, unless I misunderstand you. Unless its not actually zfs encryption but something else?
- chungy 9y agoIt's only supported encryption in the same way Linux does: by doing a lower-level block device encryption. There's a lot of disadvantages to it, and native ZFS dataset encryption would be nicer.
- gruturo 9y agoFree(BSD|NAS) so far create a "normal" ZFS on top of an encrypted block device, produced via the cryptographic GEOM provider geli ( https://www.freebsd.org/cgi/man.cgi?geli(8) https://www.freebsd.org/cgi/man.cgi?geli(8) ) This instead is ZFS doing the actual encryption on a normal block device.
- ryao 9y agoYes. The plan is for this tone the same across all OpenZFS platforms. How long it takes to be adopted by the others is another story though.
- mavhc 9y agoCan I zfs send encrypted incremental snapshots to a remote server and have them merged without ever having the remote server decrypt anything? That would be awesome to backup to an untrusted remote computer.
- binwiederhier 9y agoYes.
- mafro 9y agoWithout having read anything of the implementation.. That is little short of astounding.
- funkaster 9y agoyes, it's at the end of the post, with a few cases of how recv and send work.
- DCKing 9y agoI almost cannot wait for this. Native file system encryption will be so good, because it removes the ugly/inelegant layer of indirection with Geli or LUKS, and it allows stuff like encrypted ZFS send and receive. It allows you to switch on and off encryption dynamically. That will allow you to use services like rsync.net without having to place as much trust their internal security practices. It's great they're taking their time, but I've been looking forward to a stable release since its announcement almost a year ago :)
- cmurf 9y agoUncertain that it enables encrypted send and receive. It very likely decrypts to get it into the send format, and then is reencrypted on the receive side. Those are separate file systems so it's not certain that they share, or even should share, the same encryption key. Update: Ahh well it helps to whole article before posting. Both encrypted and decrypted send/receive are possible. But this also suggests multiple keys per pool, a key per file system I guess?
- tcaputi 9y agoYes, there are per filesystem / zvol keys and the raw encrypted data is sent over the wire along with the encrypted keys.
- rexicus 9y agoIt's a reasonable indirection, putting encryption into the file system is controversial, see NTFS for the pitfalls.
- AdmiralAsshat 9y agoThe nice thing about LUKS for the end-user, however, is that it's well-supported in most Linux distros these days, and the end-user can turn it on within the graphical installer literally by checking a box, "Encrypt my Hard-drive." When ZFS gets there, I will probably switch to it. Until then, however, I find manually partitioning volumes on the command-line to be terrifying, and I imagine I'm not alone.
- bfrog 9y agoI wish ZFS were in tree :-)