16 ms·
OpenZFS Merged to FreeBSD
- google234123 6y agoThe fact that there are still 3 main BSDs being developed in parallel seems odd to me. I imagine that they could accomplish much more if all that effort was focused on a single OS.
- _jal 6y agoThey have rather different goals.
- dman 6y agoThe fact that 7.6 billion people live in parallel seems odd to me. I imagine humanity could achieve a lot more if every aspect of human identity was put into one individual.
- deleted 6y ago[deleted]
- Rebelgecko 6y agoThe same could be said to the nth degree for Linux. In fact I think you could make the argument that the differences between the 3 most popular BSDs are way more significant than the differences between the 3 most popular Linuces
- eatbitseveryday 6y agoAren't the kernels amongst the BSDs different enough that effort is split? With GNU/Linux, the variations is much more in the packaging than the kernels, no? Couldn't I download a vanilla kernel, compile it, and boot any Linux distro with it?
- zymhan 6y agoYes but part of that is inherent in what BSD considers a kernel as well.
- spijdar 6y agoWell, yes/no. There are different ways to configure the kernel, and some distros/userlands require specific kernel features which may or may not be default. Your general point is still true, though. They are different enough that some of the work is split, however there's significant code sharing in things like device drivers and the like. Ultimately each being very different is usually seen as a positive. While the <n> Linux distros are spending significant hours optimizing essentially the same userland for minor differences in use cases, the different BSDs have very different goals and results. OpenBSD has an emphasis on making a lean and consistent base system with comprehensive documentation, NetBSD has excellent tooling for cross-compiling, and a comparatively more modular kernel that lends itself to experiments like rump-kernels. I'm less familiar with FreeBSD but it's tended to emphasize features like ZFS and its own robust container solution. In some cases these features are contradictory, as OpenBSD would be (I imagine) very reluctant to merge NetBSD's in-kernel Lua module support, or NetBSD's multithreaded "high performance" implementation of the firewall pf, "npf", in favor of its own much simpler and more secure pf. OTOH NetBSD has a stronger emphasis on backwards compatibility which somewhat limits their ability to make broad changes to the userland as OpenBSD has. Then, one is much more likely to be able to run old NetBSD binaries on new kernels. :) Each has its pros/cons, and attracts its own users and developers. The distinguishing thing about *BSDs is the base userland and kernel are deeply intertwined, giving each more freedom to deeply tie userland tools to special kernel functionality. There are pros and cons to this too, but this means that, for example, one can use "ifconfig" to connect to WPA2 WiFi networks on OpenBSD with no other tools required, as the ifconfig userland is always released alongside a matching kernel. The various net-util packages shipped for linux have suffered from having to operate as "external" tools interfacing with a range of kernel versions. (of course, now the situation is better with ip/iw, but it sure took long enough)
- Lammy 6y agoIt looks like OpenZFS still has xattr support which is the one Illumos ZFS feature I always wanted on my FreeBSD systems. Hype! :D
- codetrotter 6y agoAwesome work by all parties involved in this :D
- FullyFunctional 6y agoThis is very awesome. It makes replication (via zfs send) between FreeNAS and Linux better. Native encryption should also make zfs cloud replication much simpler.
- stiray 6y agoWhat really worries me here is compatibility with older pools.I hope they have thought about that as freebsd implementation is here since 2007.
- yjftsjthsd-h 6y agoIt's my understanding that ZFS is backwards compatible forever; old pools are on old versions or have fewer feature flags, but they can still be used without issue.
- stiray 6y agoYes in theory they are but currently syncing pools between linux and freebsd zfs is not possible so I would imagine there are some differences. I have around 12Tb of zraided disk space on freebsd and I hope I will be able to upgrade eventually.
- moonchild 6y agoEh? I currently share a zpool between linux and freebsd, and it works fine without issue.
- stiray 6y agoThank you for the info, I wasn't aware of that, the last time I have tryed it didt work (1-2 years back)
- deleted 6y ago[deleted]
- magicalhippo 6y agoZFS is backwards compatible, not forwards compatible. If you create the pool on pre-OpenZFS FreeBSD, you can use it on Linux, but not vice versa. I just did this with FreeNAS (based on FreeBSD 11) and Raspbian.
- ggm 6y agoIt is amazing that two disparate OS communities can take a shared codebase and integrate it deeply into their structure. There are many places where it would be almost impossible to achieve this kind of share. Witness the 'compatability library' work which exists to avoid ABI blackholes running one platforms code on another platform. BTW I have been able to carry FreeBSD ZFS LUN on a NetAPP and mount them on Debian ZFS, and vice-versa, since I started using ZFS. What we got here is better code alignment but with qualifications, ZFS has always been a common platform (with fringing differences) I never had illumos or Solaris, I cannot comment on how well it would have worked.
- cryptonector 6y agoTwo?? Three. More!
- kev009 6y agoBrian Behlendorf and Matt Macy are both fantastic engineers, other people contributed but this is ultimately the kind of thing that must be led by a small number of (in this case two) extremely senior engineers working together for a common outcome with raw technical ability to get everything off the ground and prove it works; and refined leadership skills to stick around til it's actually done!
- nosequel 6y agoMatt (Kip) Macy also showed "refined" leadership when he > told workers in September 2006 to cut the beams that supported (his tenant's) apartment's floor... They also shut off Morrow's electricity, cut his phone line and had workers saw a hole in his living room floor from below, prosecutors said. So yes, he has really refined leadership qualities! https://www.sfgate.com/bayarea/article/S-F-landlords-charged-with-tenant-terror-3217848.php https://www.sfgate.com/bayarea/article/S-F-landlords-charged...
- bjl 6y agoDamn what a shitstain. He should be in prison.
- FooBarWidget 6y agoHow come Linux has licensing issues with ZFS, but the BSDs don't?
- mtzet 6y agoThe ZFS license (CDDL) is (arguably) incompatible with GPL. This is not a problem for FreeBSD since it’s not using GPL.
- qalmakka 6y agoBecause the CDDL (by design, as inherited from the MPL 1.x) is not compatible with the GPL, due to the fact that both try to enforce their own copyleft without allowing relicensing (i.e the MPL2 avoids this thanks to a annex clause that allows you to consider everything as covered by the GPL instead). You can't cover the ZFS modules with the GPL in the final agglomerated result (i.e. kernel + module), so it's not allowed technically by the kernel license (see the difference between the LGPL and the GPL for more info). In practice no one ever questioned this in court, so who knows what the final result would be. FreeBSD uses a super weak permissive BSD license instead, so there is no conflict because it leaves whatever code you're trying to add to it alone, without trying to enforce any additional terms on it.
- trasz 6y agoThe issues are caused by GPL, while BSD is, well, BSD licensed.
- IntelMiner 6y agoSince Linux came first (and was GPL licensed first), it's worth remembering that ZFS was intentionally made incompatible with the GPL via the "CDDL"
- nix23 6y ago>ZFS was intentionally made incompatible with the GPL via the "CDDL" That was said by one lousy youtube comment without any explanation, Ubuntu has no problems with it...and i bet they have lawyers, i think it was in that vid: https://www.youtube.com/watch?v=-zRN7XLCRhc https://www.youtube.com/watch?v=-zRN7XLCRhc It's also crazy, nearly no distro has a problem shipping ultra-proprietary software (firmware), but has a big discussion about CDDL, no one wants ZFS in the Kernel tree, imagine Linus has control over it and has no clue was ZFS even is: https://linux.slashdot.org/story/20/01/19/0059251/what-linus-torvalds-gets-wrong-about-zfs https://linux.slashdot.org/story/20/01/19/0059251/what-linus...
- williesleg 6y agoZfs does not work well on sparse volumes, it loves new blocks. That's why you don't see it much in a VM or on storage that is sparse (go nimble, NetApp, etc) Also it's pretty much impossible to grow a zfs pool and there are no consistency check tools or repair tools. So when you actually do get corruption, you lose it all. Might have been a breakthrough around 2000 but it's time for something new.
- magicalhippo 6y ago> Also it's pretty much impossible to grow a zfs pool and there are no consistency check tools or repair tools. This is flat out wrong. First you can trivially grow a zpool by adding another vdev to it. If you can't do that, you can grow a vdev, and hence the zpool using that vdev, by swapping the disks for larger disks one at a time. Once all the physical disks in the vdev are of the new, larger size, the new space will become available automatically. Also there's absolutely a consistency check and repair tool built in. ZFS computes and stores checksums for all blocks, even when you use a single disk!. It also verifies the checksums when reading blocks. In addition you can trigger a verification of all the blocks by running a scrub command, which simply issues a read for all the blocks. > So when you actually do get corruption, you lose it all. Hardly. Not only does ZFS have the above, it also stores multiple copies of important metadata blocks (three copies by default), and it even takes effort to spread those blocks out on the physical disks to reduce the chance of them getting corrupted as much as possible. I'm not saying ZFS is the perfect filesystem. But it's definitely one of the better if you care about your data.
- benlivengood 6y ago> Also it's pretty much impossible to grow a zfs pool and there are no consistency check tools or repair tools. So when you actually do get corruption, you lose it all. `zpool scrub` is the consistency check and repair tool. It's happy to detect and repair everything from bit flips to realtime `dd if=/dev/random of=/dev/ada0` of a mirrored/raid-z disk. Growing a single vdev can be impractical especially for raid-z. Adding new vdevs to a pool is straightforward.
- fomine3 6y ago> Might have been a breakthrough around 2000 but it's time for something new. Agree with it but other stable filesystems (XFS, ext4) are 1990s tech. ZFS is only available stable 2000s filesystem. Possibly Bcachefs is the future.
- srinathkrishna 6y agoQuite the natural evolution since TrueNAS has traditionally been a BSD box and they've invested a lot of time in ZFS. Happy that this will carry over to FreeBSD. Now, only if Windows would also have some sort of support ;)
- chungy 6y agohttps://github.com/openzfsonwindows/ZFSin https://github.com/openzfsonwindows/ZFSin
- hinkley 6y agoThis is only peripherally related to this great news, and I apologize if this makes me a wet blanket to ask, but I figure someone here will know the answer: I went back to read some ZFS roadmap documentation from the Before (Oracle) Times this winter, and was reminded why my outlook for ZFS was initially rosier. I was super excited about ZFS because if and when it got the ability to add new hard drives to an existing volume, then we had something I've always wanted: a straightforward way to grow a filesystem without having to tear it down and rebuild it from scratch. That was still listed in future enhancements, shortly before Sun started to fall apart. Does anyone know where we are on that? Did that functionality get painted into a corner by other design concerns? It's in that category of things that I might not use that much, but family and customers could use the hell out of and it would simplify my life just knowing it exists. I feel we are all lesser for that never making it off of the drawing board.
- kylegordon 6y agoI think it's happening soon! Been watching this for a while too... https://github.com/openzfs/zfs/pull/8853 https://github.com/openzfs/zfs/pull/8853
- hinkley 6y agoThis PR has been sitting for a long time :(
- wtallis 6y agoThis capability is why I've always regarded ZFS as great for people who buy drives by the dozen, but a fairly poor choice for consumers and hobbyists. The ability to rebalance a btrfs array and switch to and from any of the RAID modes makes it vastly easier to cobble together useful redundancy from miscellaneous drives, or to recover from a drive failure without having to immediately go out and buy a suitably large replacement drive. If btrfs RAID5/6 hadn't taken so long to get to the current barely-usable-with-caveats state, I doubt ZFS could have caught on among home Linux users.
- 6y ago
- quotemstr 6y agoZFS still doesn't do direct IO or automatic defragmentation. It represents an early realization of a good idea, one since clipsed by other tree-based filesystems. You wouldn't fly in a Wright Flyer, so why would you run ZFS?
- yjftsjthsd-h 6y ago> You wouldn't fly in a Wright Flyer, so why would you run ZFS? Because it has literally zero competitors that can even approach its feature set? Btrfs is a trash fire, bcachefs is years out, nilfs2 doesn't actually verify its checksums, ceph/gluster run largely in userspace (so you can't run them as your root fs). What fs would you like to propose that lets us do snapshots, data integrity, and compression?
- quotemstr 6y agoIn what way is btrfs a trash fire? Works fine for me.
- fomine3 6y agoAFAIK, actual O_DIRECT support is now in progress but performence is not expected to great due to impossibility of CoW optimizing. Anyway lack of O_DIRECT is not important for some use cases. https://github.com/openzfs/zfs/pull/10018 https://github.com/openzfs/zfs/pull/10018 For defrag, ZFS developers avoid to implement "block pointer rewrite" feature that's needed for defrag due to complexity. I hope BPR project start to improve features.
- PostThisTooFast 6y agoRemember when Apple was going to adopt ZFS as Mac OS's file system? Then... nothing further was heard about it.