13 ms·
Native encryption added to ZFS on Linux
- mrsteveman1 10y agoRelated pull request with more details & discussion https://github.com/zfsonlinux/zfs/pull/4329 https://github.com/zfsonlinux/zfs/pull/4329
- deleted 10y ago[deleted]
- wspeirs 10y agotcaputi works with me at Datto. We use ZFS as our main file system for some 200+ PB of storage. It is a truly amazing file system, and this change, once merged, will add another layer of security for our customers. Great stuff!
- okket 10y agoAt this scale, why ZoL and not a more 'native' ZFS with FreeBSD (or even Illumos)? Which also perform better and have more features? Is it the lack of drivers? Better availability of other 3rd party software for Linux? The package manager? Or just "more brains in Linux"/"our CTO grew up with Linux"?
- abrookewood 10y agoI thought that ZoL and ZFS+FreeBSD were largely feature comparable?
- justincormack 10y agoOther than orders of magnitude more testing and experience and usage on FreeBSD. Much code is the same, but the glue may have bugs.
- pantalaimon 10y agoWell Linux receives orders of magnitude more testing and usage than FreeBSD, so I don't see what your argument is
- fyolnish 10y agoHis argument refers to the current state of this specific component.
- pantalaimon 10y agoZFS on Linux has been around for 3 years now and is integrated in Ubuntu 16.04 - I'd argue that by now it has received more usage than the BSD version.
- Annatar 10y agoZFS on Linux will always lag in fixes and features because OpenZFS is developed on and for illumos first and foremost.
- mioelnir 10y agoYou can argue that the earth is shaped like a torus. I can argue that the moon is violet. That is not the point, and has nothing to do with the merit of the argument. Setting this aside, all usage hours are not equal. How do you compare an hour of enduser desktop usage on Ubuntu 23.57.something to a usage hour of a commercial NAS storage solution internally based on FreeBSD and ZFS? Do you think the enduser performed comparable product testing cycles? I've seen ZOL systems where the zpool claimed it was online, but contained no vdevs. `zfs list` had no datasets, but datasets where mounted and trying to read from them got your process stuck in-kernel. It just lost/forgot its devices. Up until one of the latest releases on every boot you could roll the dice by which name your pool would import the devices. Behaving differently on identical machines and setups. Personally, I am still not trusting that problem to not reappear again. ZOL is bolted on. With a large nailgun. Simple as that. At times, it feels about as integrated as pjd's original patches distributed on the FreeBSD mailinglists. And since this division is not technical, but legal, based on the license choice made 30 something years ago for Linux with regards to where the code could be exported to and what could be imported into, this situation will not resolve itself.
- Annatar 10y agoNo; run zpool upgrade -v; zfs upgrade -v on illumos and on Linux and you will discover the truth. (upgrade -v lists the versions and features instead of performing an actual upgrade.)
- gnoway 10y agoIs there a meaningful performance difference between ZFS in FreeBSD or Illumos and ZoL? Edit: this is a legitimate question. I know previously Linux-related ZFS efforts used FUSE, but ZoL is native. I assume performance should be roughly equivalent between Linux and FreeBSD, certainly.
- Annatar 10y agoBetter observability with Dtrace, mdb, iostat, and vmstat on illumos-based systems for sure. Also simple logic dictatates that if something is made on something for that certain something, it's going to run best on that something. Linux has ZFS, but it's grafted on and the illumos POSIX layer is emulated in that sense. Further, Linux's version of OpenZFS will always lag behind fixes and features in the illumos-based systems; even FreeBSD usually contains newer vesions of OpenZFS sooner than ZFS on Linux does. Linux is just "the last hole on the porting flute". Also, experience tells me that illumos and FreeBSD based systems will always perform faster than Linux with regads to ZFS but I'd have to publish a full line of benchmarks and I bet even those would get vehemently disputed because Linux is all the rage right now, so that's a lost cause: you have to try it for yourself and make up your own mind.
- makmanalp 10y agoWhat's the stability of ZFS on linux like these days? Anyone have any positive / negative experiences to share?
- aeorgnoieang 10y agoI've been running two very-lightly-used sets of pools on two old desktops for ... about a year and a half now actually and everything's gone swimmingly. Nor have I read of anything bad happening to anyone that didn't involve not having backups. All in all, lots of people are running it, lots of people seem to be having almost entirely positive experiences with it, so it seems like it's very stable.
- agumonkey 10y agoCould you give machine specs ?
- barrkel 10y ago$ head -1 /proc/meminfo MemTotal: 4046880 kB $ grep model.name /proc/cpuinfo | head -1 model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz $ sudo zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT tank 36.2T 27.2T 9.09T - 10% 74% 1.00x ONLINE - That's 10x 4TB WD Red drives in raidz2 configuration. I've been running ZFS for the past 6 years, a little over 2 years on ZoL, and 4 years before that on Nexenta (Ubuntu userland with OpenSolaris kernel). I've had 5 HDD failures over the past few years, never lost any data. I've found ZoL to have less jitter when streaming video files in particular over SMB than the Solaris implementation did. ZoL is tighter on memory though. raidz2 isn't particularly fast for random I/O (since it acts like it has a single spindle), but I don't do much random I/O. It's mostly video and local backup cache. I'll be thinking about expanding my current setup soon enough. I'll probably be aiming for a raid10 setup for better random access, and of course the new system will have more RAM, and ECC.
- agumonkey 10y ago
- knz42 10y agoSo now thanks to this there are two implementations of AES in the linux kernel. Who's responsible for ensuring they are both correct?
- simcop2387 10y agoThis is only part of the zfs on linux port, not part of the kernel proper (i.e. mainline). The kernel team is responsible for their part. and the zfs on linux team, since they have to import their own, due to licensing concerns must maintain the library they ship.
- nine_k 10y agoThe other AES implementation is not licensed and thus can not be exported for use with ZFS. (And no, it's not the USA crypto export restrictions circa 1994, it's two pieces of freely available code with different notions of freedom coded in.)
- deleted 10y ago[deleted]
- black_knight 10y agoWhy would each file system need a "native encryption"? What gains are there fromt this? Encryption seems it would be more cleanly implemented transparently underneath the file system level.
- daenney 10y agoAs noted in the PR, in this case it's a licensing issue.
- black_knight 10y agoNo, I mean why would ZFS have anything to do with encryption at all? It doesn't seem to be a filesystem issue, rather something you put underneath it (encrypting the block device, no matter what file system is on top) or on top of it (encrypting separate files).
- otterley 10y agoIt's a feature that trades off layering purity (which ZFS has never adhered to) for increased usability.
- black_knight 10y agoI see. When crypto is concerned, this kind of purity seems attractive though… Easier to reason about when using it. If I were to use ZFS’s native crypto I would have to look into the details of their implementation to understand what guarantees are provided, and where in the stack the crypto is applied etc…
- tptacek 10y agoIn fact, it's significantly harder to reason about encryption at the block layer, and not just because it's difficult to reason about the most popular construction (XTS). Among other things, you have to come up with a semantic for handling authentication failures at a sector level. What does it mean for one sector of a file to fail authentication? Have you lost the whole file? Do you fill with zeroes? How do you signal the precise error to the kernel or users?
- DashRattlesnake 10y agoSo what's the relation of this to OpenZFS? Is this currently just for the Linux port, and not yet pulled into OpenZFS for other platforms?
- emaste 10y agoThis work is being done in ZFS on Linux but will make its way back into OpenZFS/Illumos and FreeBSD.
- emaste 10y agoNote that the commit linked here is a port of the Illumos Crypto Framework (ICF), which is a dependency but is not the change that actually brings native encryption.
- mei0Iesh 10y agoI think this is most useful for cross-compatibility. Right now if a client uses FreeBSD ZFS, and you need to mount it to access project files on your Linux desktop, you can't if they used encryption. But after this is standard, you should be able to mount the same ZFS filesystem anywhere.
- mioelnir 10y agoDo you create zpools on memory sticks? Or how do you export the pool and move the device with the exported pool on it to your linux desktop?
- mei0Iesh 10y agoYes, you can use ZFS on memory sticks. You can also "zfs send | zfs recv" to copy a single projects dataset snapshot. You don't need to use actual disks though, if someone wants to send you project files you can have them either send you a file from "zfs send > snapshot.zfs", or send it over SSH, "ssh friend zfs send | zfs recv". Encryption could be an issue if for example someone uses a FreeBSD based NAS for large data files, and you want to skip the network and just access them directly from your Linux box. You can "zfs export; zfs import", but not if they used encryption. That's where I think this will be useful, because then we have one standard filesystem we can use everywhere.
- mioelnir 10y agoI am well aware of all of that. But all the zfs send/recv options can by their very definition not have the full disk encryption problem hinted at in the comment I replied to. Also, if it is easier to take your NAS offline and apart to chuck the disks into your desktop (compared exporting it over the network), then your NAS is too small. What you were looking for is a Laptop. Which leaves abusing the zpool on a memory stick as data interchange format. Most likely with copies=1, so you have to add some par2 files anyway, at which point you could simply put them on figuratively any other filesystem out there. And encrypt them with gpg/openssl etc. That way I would also not have to run a potentially maliciously crafted filesystem within my kernel.
- Infernal 10y agoAs someone who just last night set up an Ubuntu 16.04 server with the intention of using ZFS, should I wait until this hits the Ubuntu repos? Is it possible to enable this encryption on an existing filesystem?
- thehnguy 10y agoSimilar questions here. It's unclear to me how I can integrate this into my system and how to go about it. I'd hate to have to blow away my zpool to do this. My zpool isn't very full. Maybe I can make a new dataset that's encrypted and move everything from the unencrypted dataset to the encrypted dataset. Of course, being able to encrypt in place would be most ideal, but I'll settle for it being a painless operation that doesn't require blowing away a zpool.
- jvehent 10y agoThat is a terrifying amount of crypto code. Has anyone audited this, or plan to?
- jlgaddis 10y agoIt was Signed-off-by two devs @llnl.gov so there's no need to worry, the U.S. Government is looking out for us. /s
- zxv 10y agonative encryption with AES-NI support sounds awesome. When is likely to make it into ubuntu repos?
- jlgaddis 10y agoSlightly off-topic but if anyone has any resources on performing a clean (preferably, Ubuntu or Arch) Linux "root on ZFS" installation, please share. I followed the instructions for Ubuntu 16.04 on the github.com/zfsonlinux wiki [0] a while back and (encountered a few little issues along the way but) got it working, although I experienced some MAJOR performance problems so something wasn't quite right (exact same hardware is blazing fast when running FreeBSD). I can't imagine it was just "how things are" with regard to the current state of ZFS on Linux (or Ubuntu specifically) -- it was like someone hit the laptop's "pause" button a couple of times per minute. [0]: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-...
- imsofuture 10y agoThis sounds crazy, and is, but following those instructions verbatim with a 16.04 install I managed to brick a new laptop. Yeah...
- iso-8859-1 10y agoDefine "brick". You flashed the wrong BIOS?
- pantalaimon 10y agoPossibly deleted the efi partition
- vmp 10y agoOr "rm -Rf /" with a writable /sys/firmware/efi/efivars/ mount. https://github.com/systemd/systemd/issues/2402 https://github.com/systemd/systemd/issues/2402
- the_why_of_y 10y agoThis problem was fixed half a year ago, surely Ubuntu 16.04 should have a backport of the relevant commits? https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0389075ecfb6231818de9b0225d3a5a21a661171 https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....
- ryao 10y agoThis is not native encryption that was committed. It is just the kernel cryptography framework required for native encryption. Native encryption comes next.
- mrsirduke 10y ago> We cannot use the Linux kernel's built in crypto api because it is only exported to GPL-licensed modules. What a strange choice by the Linux kernel.
- deathanatos 10y agoWhy is this a strange choice? Isn't most of the kernel GPL? (and thus, a non-GPL component can't link to the GPL part, as that would be a violation of the GPL, would it not?)
- cyphar 10y agoThe crypto code is all very internal to the kernel, so of course it will be GPL. Besides, I personally think that all kernel drivers should be GPL because they are essentially all derivative works of the kernel. The ability of module loading to get around the GPL is one of the more worrying decisions made by Torvalds.
- lmm 10y agoSo does ZFS on FreeBSD support native encryption? Can I switch my existing pool?
- nixomose 10y agoIs there really a need to make zfs your root volume? You can reinstall your root volume in a few minutes from a flash drive. What you really want is your home directory to be zfs, and just do all your work in your home directory. Saves all the grief of trying to make it your boot volume and it works just as well.
- doublerebel 10y agoThis is basically what I do. A copy of the root volume can be kept in a ZFS filesystem since the zpool can be easily mounted during any recovery.