6 ms·
I've used both ZFS on both BSD and OpenIndiana, and I've used btrfs on linux. as recently as 4.0.5 As of 4.0.5 btrfs was IMO completely unusable as a daily fil
by aidenn0 11y ago
I've used both ZFS on both BSD and OpenIndiana, and I've used btrfs on linux. as recently as 4.0.5
As of 4.0.5 btrfs was IMO completely unusable as a daily file system. Some examples of issues I ran into:
1) System became unbootable with the version of btrfs I had installed and I had to use either an older or newer kernel to recover
2) I have a periodic backup of my mailbox that runs, and when it runs my system becomes completely unusable until it completes. The same script running on zfs on bsd and with ext4 or reiser3 on linux would show I/O slowdowns, but I could still use my machine.
3) In general I would run into other minor issues and the consensus in #btrfs was that since my kernel was more than 3 months old, it was probably fixed in the latest version, and why would somebody using an experimental filesystem not be tracking mainline more closely?
[edit]
To be fair, here's some issues with ZFS:
1) Do not ever accidentally enable dedupe on commodity hardware; it will slowly consume all your RAM if you aren't on a sun server (where 64GB of RAM is a resource constrained environment), and there aren't effective ways to undo dedupe, other than copying all the data onto a different pool.
2) You can't shrink a pool. Hugely annoying, apparently non-trivial to solve.
- rsync 11y ago"To be fair, here's some issues with ZFS:" Let me add: 3) Do not allow a pool to exceed 90% capacity ... and probably don't let it exceed 85%. ZFS does not have a defrag utility and it badly needs one. You can permanently wreck zpool performance by running it up past 90% capacity - even if you later reduce capacity back down to 75-80%. You can sort of fix it if you add additional top level vdevs to the zpool, thus farming out some IO to the new set of disks, but it's still going to be performance constrained forever. The only solution is to create a new pool and export the data to it. This is unacceptable, by the way. It is not at all reasonable to require a filesystem to stay below 80% capacity (our target "full" number at rsync.net) nor is it acceptable that hitting 90% is a (performance) death sentence. When you consider that you might have already sacrificed 25% of your physical disk just for the raidz2/raidz3 overhead, being constrained to 80% means you're only using 60% or so of your physical disks that you bought. ZFS needs defrag. Badly.
- ryao 11y agoIf gang blocks are generated, you get more I/Os than necessary and an extra level of indirection, but the ZFS code base tries very hard to avoid that situation by switching to the allocator behavior at the metaslab level to best fit, such that most data written to the pool would not have gang blocks used at all. This is understandably terrible for IOPS, but it is likely the source of the performance degradation that you saw. You can probe for the zio_gang_* functions during a scrub to see if any gang blocks exist. On my pool that has exceeded 90% on multiple ocasioks, there are zero gang blocks and consequently, no permanent degradation from them. The only other problem that you might have (which tends to be caused by the order of writes rather than the fullness of the pool) is lower sequential performance from nonlinear block placement (one ZoL user measured this as cutting sequential reads in half on a pool filled with files made by bit torrent), but that is a much less severe problem, especially on solid state drives. If you want to fix placement, you can do a file copy or send/recv. The new locations should have blocks picked in sequence whenever possible.
- l1ambda 11y agoThe Reiser guys were talking about this too. I think they needed defrag and were trying to get DARPA money to implement it, but it never happened.
- cthalupa 11y agoFor ZFS to get defrag, someone with enough ability and knowledge would need to bite the bullet and go through the fairly massive and difficult task of adding in block pointer rewrite, which doesn't seem to be something anyone has been willing to do, and I've seen a lot of concerns about the actual feasibility of it from some very smart people that are knowledgeable with the codebase.
- riquito 11y agoI've been using btrfs since 2 years (Fedora) and the only problem I've ever run into was "no space left on device", solvable with a rebalance. btrfs also survived many hard poweroff.
- feld 11y agoI tried to use the mirror functionality when it was new. I tested booting with one disk missing. Errors all the way. I went into IRC and chatted with btrfs folk about the "bug". Their response? "Booting without all members of your mirror is unsupported."
- zanny 11y agoI'll pitch in with a neutral position. I've been using btrfs for four years, and in that time I've had unrecoverable fs corruption probably three times. This is on Arch, on bleeding edge kernels, where new releases are prone to regressions that break the filesystem. But there was a tangible progression from instability towards increasing stability. I haven't had one lick of issue with btrfs in about a dozen kernel releases. I'm close to saying I'd trust in a production environment, since I use it everywhere else as a daily driver, I would just use an LTS release version just to be safe. It is not all sunshine and roses, though. While Facebook employs several major btrfs developers, a lot of features that have been talked about for years still have not seen the light of day or any development whatsoever. lz4 compression, better checksum algorithms, per-subvolume encryption, online filesystem checking, and the Raid 5/6 support is still kind of garbage a year later. I worry that btrfs is suffering from a lack of interest in actually making the last legitimate pushes it needs, and code audits, and integration testing, to make it truly trustworthy. But at the end of the day checksum integrity and COW are basically a game changer for me in terms of data integrity.