6 ms·

Good luck. There is long road ahead. Both ZFS and Btrfs were initially developed by really high caliber people and experts with good track record. ZFS had five

Good luck. There is long road ahead. Both ZFS and Btrfs were initially developed by really high caliber people and experts with good track record. ZFS had five years for full time development until release, next five years to get close to the features and stability that ZFS has now. Btrfs started 10 years ago and it's still trying to catch up.

X86BSD 9y ago
I think you think to highly of the btrfs group. There not bad people but Filesystems are HARD to get right and they simply are not in the same league as the zfs developers were. The well known raid 5 issue btrfs had shows a prime example that the btrfs simply is not in the same class as zfs' design. I wouldn't compare the two.
- eriknstr 9y ago
  I use ZFS under FreeBSD but I am wondering what the future of ZFS looks like. I went to the OpenZFS website (http://open-zfs.org/wiki/Main_Page http://open-zfs.org/wiki/Main_Page) to try and get a feel for the amount of work currently going on. They have some videos and slides, they have a mailing list. The mailing list is mostly full of messages via GitHub but at least that lead me to https://github.com/openzfs/openzfs https://github.com/openzfs/openzfs which I was previously unaware of and which I had not found when trying to look for an OpenZFS specific repository. As far as I have come to understand, the goal of the OpenZFS project is to merge changes made to ZFS by Illumos, FreeBSD, ZFS-on-Linux and other projects. Their videos might answer this but I wish their website had a clear and simple overview of what has been done. Most of what I can find on their wiki is various ideas for things that they want to do, but I can't know if that means that the OpenZFS project is mostly talk and not so much action, or if it's just that they have done a lot of those things but because they are busy doing they don't have time to write about it. Could be that they have some pages on the wiki I haven't seen also but in that case I think it should be organized better. I know that Joyent picked up several highly skilled software engineers and programmers that used to work for Sun, and that their SmartOS operating system builds on code descendant from OpenSolaris and that ZFS is as integral on SmartOS as it was on Solaris, perhaps even more so on SmartOS. I have not seen mention of SmartOS on the OpenZFS wiki though. All in all I feel that ZFS is still being actively developed and maintained by many people. But to what extent they are able to cooperate as much as a lot of them seem to want to I would like to know. It would be a shame if btrfs overtook ZFS simply because the development of ZFS was too fragmented :/
  - thatcks 9y ago
    The various Illumos distributions routinely upstream their changes into the master Illumos repository ( https://github.com/illumos/illumos-gate https://github.com/illumos/illumos-gate). I know that ZFS on Linux (https://github.com/zfsonlinux/zfs https://github.com/zfsonlinux/zfs) routinely integrates those changes, as well as making some of its own and I believe pushing them upstream to Illumos. When I've looked at FreeBSD commit logs, they too are pulling in ZFS changes from Illumos on a frequent basis (but I know less about any independent development and upstreaming). My strong impression is that the ZFS code base is a sufficiently big and tangled thing that no one wants it to fragment. With bug-fixes and improvements happening in Illumos, ZoL and FreeBSD both want to be able to incorporate them on a regular basis and to push their own changes upstream to reduce the maintenance burden of carrying those changes.
    - X86BSD 9y ago
      Illumos is the repo of record. Everyone of the above OS distributions feeds changes upstream keeping zfs in sync between platforms. No one relying on zfs wants forks or incompatibilities. That's one of zfs main selling points, that you can yank a pool from FreeBSD and shove them into illumos and vice Versa.
- cmurf 9y ago
  Btrfs started as a free and open source project from day 1, and people started using it from the moment it was merged mainline which was something like a year. ZFS was started internally at Sun with a bunch of resources dedicate from people who had been involved in and thinking about storage devices and file systems and their myriad problems for a long time well before they started ZFS. And the cat wasn't out of the bag to end users for about 4 years, during which time there wasn't the contributor pile on effect. Btrfs had to contend with some balancing act of not saying "no thanks" to patches, meanwhile those contributions very likely did clutter up the code every bit as much as allowing the contribution aided in hyping the project rather than turning people away from contributing at all. As for raid56, ZFS emerged before cluster file systems like Ceph and Gluster were even conceived. The companies that need to store tons of data, and fund various storage related projects don't care nearly as much as they once did about raid56. They can just replicate the data elsewhere using Gluster and if a whole brick, be it XFS or Btrfs implodes, they can just make a new brick and the data gets replicated again. Anyway, ZFS and Btrfs are really not comparable even though on the surface they seem to do really similar things.
joshbaptiste 9y ago
Exactly, BcacheFS, HAMMER2 are other file-systems that are being worked on by some REALLY good programmers for years and years to come. It seems like a file-system takes longer to stabilize than a kernel.
- laumars 9y ago
  > It seems like a file-system takes longer to stabilize than a kernel. For good reason in my opinion. An unstable kernel will cause application glitches, unnecessary slowness or OS crashes. They are annoying but not persistent (ie a reboot and you can carry on for a bit). However an unstable file system are persistent and thus could destroy all of your data forcing you to recover from backups (assuming you're diligent enough to keep tested backups). Plus file systems still have to deal with buggy consumer hardware and other similar edge cases (eg storage devices, power failures, etc) just like a kernel would.
  - lordnaikon 9y ago
    It is even worse. You can recover from backups but that implies that you already know at which point you destroyed your data and how fast you discovered that – if that ever happens. Imagine you're collecting data over a substantial period of time (maybe do calculations based on it that influence how you're collecting further data) and you're very smart and backup all your data let's say every day. The problem is if your data gets defects due to the file-system it is very unlikely to detect that. You can run your system for like a year without apparent problems whatsoever. Lets say that error accumulates and you eventually detect this by accident. Your backups are pretty much worthless ...
- dom0 9y ago
  I've had some discussions with file system developers and a developer of a distributed file system. The sheer number of failure cases that need to be handled, because they are hit in production systems, is simply staggering.
bluejekyll 9y ago
Has anyone done an analysis of the number of logic bugs in ZFS and btrfs vs. memory safety or concurrent updates to memory, etc.? Also, I'll point out that Apple just dropped a new FS on millions of devices, with no issues... that was developed in 3 or 4 years. I'm still blown away that they pulled that off.
- pas 9y ago
  They have full control of that environment though. So they tested it on a few hundred devices, and by that exhausted all the possible configurations. And when all worked, they knew they have a pretty good indicator for successful deployment on all those millions of devices.
  - dom0 9y ago
    It's also used only with their specific flash storage devices.
  - bluejekyll 9y ago
    Why downplay it? It's an impressive feat, regardless of the differences in target devices. Even with the hardware configurations well known, the fact that it was done at such a large scale successfully means that even unusual edge conditions didn't crop up. This shouldn't be downplayed, it actually speaks to why it's important to have incremental stages of software delivery. First target highly constrained environments (iOS, Watch OS, tvOS), then work on the more difficult and less constrained general computing environment.
    - dom0 9y ago
      It's absolutely an impressive feat to pull this off. But it's not quite the same problem as building a robust general-purpose FS for a diverse ecosystem.
      - derefr 9y ago
        True; and yet, it feels like there is much low-hanging fruit left in filesystens that are just built for specific vertically-integrated use-cases. A NAS hardware-appliance company, for example, could likely pull off something similar to what Apple did, and to great benefit.
poopsicle 9y ago
High caliber people need to get paid. Sun invested heavily in zfs. No one (and by that I mean organizations with resources) is really giving btrfs much attention.

X86BSD 9y ago

I think you think to highly of the btrfs group. There not bad people but Filesystems are HARD to get right and they simply are not in the same league as the zfs developers were. The well known raid 5 issue btrfs had shows a prime example that the btrfs simply is not in the same class as zfs' design. I wouldn't compare the two.

eriknstr 9y ago
I use ZFS under FreeBSD but I am wondering what the future of ZFS looks like. I went to the OpenZFS website (http://open-zfs.org/wiki/Main_Page http://open-zfs.org/wiki/Main_Page) to try and get a feel for the amount of work currently going on. They have some videos and slides, they have a mailing list. The mailing list is mostly full of messages via GitHub but at least that lead me to https://github.com/openzfs/openzfs https://github.com/openzfs/openzfs which I was previously unaware of and which I had not found when trying to look for an OpenZFS specific repository. As far as I have come to understand, the goal of the OpenZFS project is to merge changes made to ZFS by Illumos, FreeBSD, ZFS-on-Linux and other projects. Their videos might answer this but I wish their website had a clear and simple overview of what has been done. Most of what I can find on their wiki is various ideas for things that they want to do, but I can't know if that means that the OpenZFS project is mostly talk and not so much action, or if it's just that they have done a lot of those things but because they are busy doing they don't have time to write about it. Could be that they have some pages on the wiki I haven't seen also but in that case I think it should be organized better. I know that Joyent picked up several highly skilled software engineers and programmers that used to work for Sun, and that their SmartOS operating system builds on code descendant from OpenSolaris and that ZFS is as integral on SmartOS as it was on Solaris, perhaps even more so on SmartOS. I have not seen mention of SmartOS on the OpenZFS wiki though. All in all I feel that ZFS is still being actively developed and maintained by many people. But to what extent they are able to cooperate as much as a lot of them seem to want to I would like to know. It would be a shame if btrfs overtook ZFS simply because the development of ZFS was too fragmented :/
- thatcks 9y ago
  The various Illumos distributions routinely upstream their changes into the master Illumos repository ( https://github.com/illumos/illumos-gate https://github.com/illumos/illumos-gate). I know that ZFS on Linux (https://github.com/zfsonlinux/zfs https://github.com/zfsonlinux/zfs) routinely integrates those changes, as well as making some of its own and I believe pushing them upstream to Illumos. When I've looked at FreeBSD commit logs, they too are pulling in ZFS changes from Illumos on a frequent basis (but I know less about any independent development and upstreaming). My strong impression is that the ZFS code base is a sufficiently big and tangled thing that no one wants it to fragment. With bug-fixes and improvements happening in Illumos, ZoL and FreeBSD both want to be able to incorporate them on a regular basis and to push their own changes upstream to reduce the maintenance burden of carrying those changes.
  - X86BSD 9y ago
    Illumos is the repo of record. Everyone of the above OS distributions feeds changes upstream keeping zfs in sync between platforms. No one relying on zfs wants forks or incompatibilities. That's one of zfs main selling points, that you can yank a pool from FreeBSD and shove them into illumos and vice Versa.
cmurf 9y ago
Btrfs started as a free and open source project from day 1, and people started using it from the moment it was merged mainline which was something like a year. ZFS was started internally at Sun with a bunch of resources dedicate from people who had been involved in and thinking about storage devices and file systems and their myriad problems for a long time well before they started ZFS. And the cat wasn't out of the bag to end users for about 4 years, during which time there wasn't the contributor pile on effect. Btrfs had to contend with some balancing act of not saying "no thanks" to patches, meanwhile those contributions very likely did clutter up the code every bit as much as allowing the contribution aided in hyping the project rather than turning people away from contributing at all. As for raid56, ZFS emerged before cluster file systems like Ceph and Gluster were even conceived. The companies that need to store tons of data, and fund various storage related projects don't care nearly as much as they once did about raid56. They can just replicate the data elsewhere using Gluster and if a whole brick, be it XFS or Btrfs implodes, they can just make a new brick and the data gets replicated again. Anyway, ZFS and Btrfs are really not comparable even though on the surface they seem to do really similar things.

joshbaptiste 9y ago

Exactly, BcacheFS, HAMMER2 are other file-systems that are being worked on by some REALLY good programmers for years and years to come. It seems like a file-system takes longer to stabilize than a kernel.

laumars 9y ago
> It seems like a file-system takes longer to stabilize than a kernel. For good reason in my opinion. An unstable kernel will cause application glitches, unnecessary slowness or OS crashes. They are annoying but not persistent (ie a reboot and you can carry on for a bit). However an unstable file system are persistent and thus could destroy all of your data forcing you to recover from backups (assuming you're diligent enough to keep tested backups). Plus file systems still have to deal with buggy consumer hardware and other similar edge cases (eg storage devices, power failures, etc) just like a kernel would.
- lordnaikon 9y ago
  It is even worse. You can recover from backups but that implies that you already know at which point you destroyed your data and how fast you discovered that – if that ever happens. Imagine you're collecting data over a substantial period of time (maybe do calculations based on it that influence how you're collecting further data) and you're very smart and backup all your data let's say every day. The problem is if your data gets defects due to the file-system it is very unlikely to detect that. You can run your system for like a year without apparent problems whatsoever. Lets say that error accumulates and you eventually detect this by accident. Your backups are pretty much worthless ...
dom0 9y ago
I've had some discussions with file system developers and a developer of a distributed file system. The sheer number of failure cases that need to be handled, because they are hit in production systems, is simply staggering.

bluejekyll 9y ago

Has anyone done an analysis of the number of logic bugs in ZFS and btrfs vs. memory safety or concurrent updates to memory, etc.? Also, I'll point out that Apple just dropped a new FS on millions of devices, with no issues... that was developed in 3 or 4 years. I'm still blown away that they pulled that off.

pas 9y ago
They have full control of that environment though. So they tested it on a few hundred devices, and by that exhausted all the possible configurations. And when all worked, they knew they have a pretty good indicator for successful deployment on all those millions of devices.
- dom0 9y ago
  It's also used only with their specific flash storage devices.
- bluejekyll 9y ago
  Why downplay it? It's an impressive feat, regardless of the differences in target devices. Even with the hardware configurations well known, the fact that it was done at such a large scale successfully means that even unusual edge conditions didn't crop up. This shouldn't be downplayed, it actually speaks to why it's important to have incremental stages of software delivery. First target highly constrained environments (iOS, Watch OS, tvOS), then work on the more difficult and less constrained general computing environment.
  - dom0 9y ago
    It's absolutely an impressive feat to pull this off. But it's not quite the same problem as building a robust general-purpose FS for a diverse ecosystem.
    - derefr 9y ago
      True; and yet, it feels like there is much low-hanging fruit left in filesystens that are just built for specific vertically-integrated use-cases. A NAS hardware-appliance company, for example, could likely pull off something similar to what Apple did, and to great benefit.

poopsicle 9y ago

High caliber people need to get paid. Sun invested heavily in zfs. No one (and by that I mean organizations with resources) is really giving btrfs much attention.