11 ms·

Native Encryption for ZFS on Linux

169 points by funkaster 9y ago

adrianscott 9y ago
s/Merkel/Merkle/
- jlgaddis 9y ago
  Thank you so much for your wonderful contribution to this discussion.
  - the_common_man 9y ago
    What's wrong with pointing out a typo?
    - deleted 9y ago
      [deleted]
    - jlgaddis 9y ago
      What purpose did it serve? The typo was in a commit message. Assuming that the committer ever even sees it -- most submissions here are not submitted by the original author -- he is certainly not going to go back and fix the typo in his commit message. Besides, in 99% of "typo instances" it is quickly obvious that a typo was made, everyone understands what it should be, and continues on. If it's a typo that actually results in a real problem then by all means point it out; otherwise it just adds to the noise (such as in this case, where it was merely a transposition of two characters).
belovedeagle 9y ago
The title of this HN post should mention that this is ZFSonLinux. This is good news, but I'll definitely want to wait a good long while before enabling this in production. Yes, officially zfs isn't good enough to use in production anywhere even without shiny new features, but I reckon zfs as-is is better than some other filesystem.
- josteink 9y ago
  By the ZFS proponents own standard for "mature", I guess I should still wait a decade to use this, right? ;)
  - jlgaddis 9y ago
    Depends on a). how much you value your data and/or b). how much you trust your backups.
ComputerGuru 9y ago
OK, I'm curious: why are people shoehorning the filesystem to fit the OS rather than carefully picking the OS that makes the most sense for the application it's being used for? We originally ran OpenSolaris _just for ZFS_. Then we switched to FreeBSD when OpenSolaris/OpenIndiana dead-ended. We had a production server running PHP and we were sick of PHP5, so we switched to Linux because HHVM on FreeBSD was a joke. PHP 7 came out and obviated HHVM (from a performance perspective), and so our latest PHP deployments are back on FreeBSD. If your data is valuable to you (and presumably it is and that's why you're looking at ZFS), why are you not on FreeBSD in the first place? Same story with ASP.NET - tried deploying on Mono way back before .NET Core was even an idea and realized the futility of it and switched a server farm to Windows _because it just wasn't worth it._ PHP on Windows? Same story - not production ready, move to Linux. (This isn't to disparage ZfsOnLinux, which I think is a great effort and laudable if only for home user purposes. Instead, this is a question for sysadmins on HN who are using ZoL.. .why?)
- detaro 9y ago
  I'm curious in what way ZoL is "shoehorned to fit the OS"? Sure, a port adds potential for error, but is the technical difference between the interfaces so large?
  - ComputerGuru 9y ago
    The ZoL code is not the shoehorning, it's the fact that you're doing so via a community effort to go around the official stance of the kernel developers who've explicitly opted to not support this FS at this time; meaning you're missing out on all the normal development and QA processes and the benefits thereof. There's real value in having the the actual OS community designing the filesystem as well, their experience cannot be discounted. This isn't just applicable to Linux; we have a strict policy on using only in-kernel filesystems on FreeBSD as well after some poor experiences (_in production_) with aufs-esque ports to FreeBSD. They all look shiny and nice on the outside, and even stress test OK, but when you release them to the masses that's when the shit hits the fan and you realize they're just not designed to the same specifications as the rest of the OS components.
williamstein 9y ago
The rate of work and breadth of contributors to ZFSOnLinux is pleasantly suprising and impressive: https://github.com/zfsonlinux/zfs/graphs/contributors https://github.com/zfsonlinux/zfs/graphs/contributors
- dom0 9y ago
  Things that happen when you employ folks full-time to work on something.
d33 9y ago
Why would one choose it over dm-crypt? What are the advantages?
- veeti 9y ago
  Authenticated encryption, for one.
- e12e 9y ago
  Personally, I think the best thing is having (finally!) serious cross-platform encrypted drive support in software. (Yes hw encryption has its benefits, but like with software raid - the flexibility of software is great). Potentially this could also be easier to use for external disks than unlock, detect volumes, mount (or unlock zfs mount...). In theory, having compression and encryption handled by one codebase might open the door to a safe/clearly delineated tradeoffs of enabling both.
  - X86BSD 9y ago
    This to me, is one of the GREATEST advances since Unix was created. Seriously, how !@#!@# amazing is it to be able to yank a pool from illumos, Linux, FreeBSD etc and swap it into a box of another OS supporting ZFS and simply import it wether its encrypted or not but especially encrypted! I don't know if most of you are old enough to remember the absolute HELL of dealing with tape drives, and UFS to trying to get data recovered or migrated before ZFS. It usually resulted in tears.
    - grahamjperrin 9y ago
      ZFS - native encryption, resizing, bleaching | The FreeBSD Forums <https://forums.freebsd.org/threads/56869/> https://forums.freebsd.org/threads/56869/> > I hope that FreeBSD will be not too far behind. …
mp3geek 9y ago
Does ZFS use the native compression libs in the linux kernel?
akerro 9y ago
Only by looking at issues in github repo you can tell it's far, far away from being stable, even for homelabs.
XorNot 9y ago
I'm curious how long it'll take for support to land in grub for this (or how much work it's likely to be).
coretx 9y ago
It might not be fair but reading a .gov address listed as a reviewer does not exactly inspire trust. :'(
- _joel 9y ago
  Why not? It's LLNL, who are heavy ZFS users and have been at the forefront of large datasets since they were a thing in computing.
  - coretx 9y ago
    Yet also advertise to be into counterterrorism, defense, and intelligence operational support. How many dual_ec_drbg do we need before we stop trusting those people ?
    - _joel 9y ago
      I think you're confusing the NSA with LLNL

169 points by funkaster 9y ago

adrianscott 9y ago

s/Merkel/Merkle/

jlgaddis 9y ago
Thank you so much for your wonderful contribution to this discussion.
- the_common_man 9y ago
  What's wrong with pointing out a typo?
  - deleted 9y ago
    [deleted]
  - jlgaddis 9y ago
    What purpose did it serve? The typo was in a commit message. Assuming that the committer ever even sees it -- most submissions here are not submitted by the original author -- he is certainly not going to go back and fix the typo in his commit message. Besides, in 99% of "typo instances" it is quickly obvious that a typo was made, everyone understands what it should be, and continues on. If it's a typo that actually results in a real problem then by all means point it out; otherwise it just adds to the noise (such as in this case, where it was merely a transposition of two characters).

belovedeagle 9y ago

The title of this HN post should mention that this is ZFSonLinux. This is good news, but I'll definitely want to wait a good long while before enabling this in production. Yes, officially zfs isn't good enough to use in production anywhere even without shiny new features, but I reckon zfs as-is is better than some other filesystem.

josteink 9y ago
By the ZFS proponents own standard for "mature", I guess I should still wait a decade to use this, right? ;)
- jlgaddis 9y ago
  Depends on a). how much you value your data and/or b). how much you trust your backups.

ComputerGuru 9y ago

OK, I'm curious: why are people shoehorning the filesystem to fit the OS rather than carefully picking the OS that makes the most sense for the application it's being used for? We originally ran OpenSolaris _just for ZFS_. Then we switched to FreeBSD when OpenSolaris/OpenIndiana dead-ended. We had a production server running PHP and we were sick of PHP5, so we switched to Linux because HHVM on FreeBSD was a joke. PHP 7 came out and obviated HHVM (from a performance perspective), and so our latest PHP deployments are back on FreeBSD. If your data is valuable to you (and presumably it is and that's why you're looking at ZFS), why are you not on FreeBSD in the first place? Same story with ASP.NET - tried deploying on Mono way back before .NET Core was even an idea and realized the futility of it and switched a server farm to Windows _because it just wasn't worth it._ PHP on Windows? Same story - not production ready, move to Linux. (This isn't to disparage ZfsOnLinux, which I think is a great effort and laudable if only for home user purposes. Instead, this is a question for sysadmins on HN who are using ZoL.. .why?)

detaro 9y ago
I'm curious in what way ZoL is "shoehorned to fit the OS"? Sure, a port adds potential for error, but is the technical difference between the interfaces so large?
- ComputerGuru 9y ago
  The ZoL code is not the shoehorning, it's the fact that you're doing so via a community effort to go around the official stance of the kernel developers who've explicitly opted to not support this FS at this time; meaning you're missing out on all the normal development and QA processes and the benefits thereof. There's real value in having the the actual OS community designing the filesystem as well, their experience cannot be discounted. This isn't just applicable to Linux; we have a strict policy on using only in-kernel filesystems on FreeBSD as well after some poor experiences (_in production_) with aufs-esque ports to FreeBSD. They all look shiny and nice on the outside, and even stress test OK, but when you release them to the masses that's when the shit hits the fan and you realize they're just not designed to the same specifications as the rest of the OS components.

williamstein 9y ago

The rate of work and breadth of contributors to ZFSOnLinux is pleasantly suprising and impressive: https://github.com/zfsonlinux/zfs/graphs/contributors https://github.com/zfsonlinux/zfs/graphs/contributors

dom0 9y ago
Things that happen when you employ folks full-time to work on something.

d33 9y ago

Why would one choose it over dm-crypt? What are the advantages?

veeti 9y ago
Authenticated encryption, for one.
e12e 9y ago
Personally, I think the best thing is having (finally!) serious cross-platform encrypted drive support in software. (Yes hw encryption has its benefits, but like with software raid - the flexibility of software is great). Potentially this could also be easier to use for external disks than unlock, detect volumes, mount (or unlock zfs mount...). In theory, having compression and encryption handled by one codebase might open the door to a safe/clearly delineated tradeoffs of enabling both.
- X86BSD 9y ago
  This to me, is one of the GREATEST advances since Unix was created. Seriously, how !@#!@# amazing is it to be able to yank a pool from illumos, Linux, FreeBSD etc and swap it into a box of another OS supporting ZFS and simply import it wether its encrypted or not but especially encrypted! I don't know if most of you are old enough to remember the absolute HELL of dealing with tape drives, and UFS to trying to get data recovered or migrated before ZFS. It usually resulted in tears.
  - grahamjperrin 9y ago
    ZFS - native encryption, resizing, bleaching | The FreeBSD Forums <https://forums.freebsd.org/threads/56869/> https://forums.freebsd.org/threads/56869/> > I hope that FreeBSD will be not too far behind. …

mp3geek 9y ago

Does ZFS use the native compression libs in the linux kernel?

akerro 9y ago

Only by looking at issues in github repo you can tell it's far, far away from being stable, even for homelabs.

XorNot 9y ago

I'm curious how long it'll take for support to land in grub for this (or how much work it's likely to be).

coretx 9y ago

It might not be fair but reading a .gov address listed as a reviewer does not exactly inspire trust. :'(

_joel 9y ago
Why not? It's LLNL, who are heavy ZFS users and have been at the forefront of large datasets since they were a thing in computing.
- coretx 9y ago
  Yet also advertise to be into counterterrorism, defense, and intelligence operational support. How many dual_ec_drbg do we need before we stop trusting those people ?
  - _joel 9y ago
    I think you're confusing the NSA with LLNL