6 ms·

ZFS for Mac OS X Breaths New Life

70 points by nilio 16y ago

I'm not sure how many folks have been mourning the seeming loss of ZFS for Mac OS X for as long as I have... but I thought that this was not only great news, but a tremendous opportunity for folks to really pitch in and help a very worthwhile project for those who value their data! Very exciting!

melvinram 16y ago
For those (like me) who are clueless on what ZFS is, the wiki page has a good overview: In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity (protection against bit rot, etc), support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. http://en.wikipedia.org/wiki/ZFS http://en.wikipedia.org/wiki/ZFS
- ihodes 16y ago
  ZFS is really cool. The best things that stood out out to me when I researched it a bit a while ago (though there is so much cool about it) are 1) more storage than you could ever possibly use–you won't run out. 2) you can add and remove drives on the fly. 3) you don't have drives you mount, you have pools you can add to and remove from (on the fly). 4) great redundancy/error-checking built in, in all forms. I'd love to see it on the Mac, officially…
  - dlsspy 16y ago
    You can add and replace, but you cannot remove drives.
    - Andrewski 16y ago
      Sure you can. You just can't shrink the size of a pool.
  - patrickgzill 16y ago
    The best feature is that you can tell it to do a "zpool scrub poolname" and it will go through the entire set of drives in that pool and check the checksums, correcting as necessary. Given that there are errors that can occur silently, ZFS can detect and correct them if they are single-bit errors.
js2 16y ago
"Breaths new life" is a bit misleading' no? From the project page: "this code project will be used to record issues and ongoing development (if any) of this extension" As well, only a single commit to the git repo since July. :(
- anywho 16y ago
  the forum is very active, especially alex blewitt. There are fairly infrequent updates, but it's definiatly being developed.
- dlsspy 16y ago
  Depends on which one you're looking at. Alex's repo has a tremendous amount of work. I've been doing little work since the initial build and installer since work picked up. I went ahead and merged mine up.
  - js2 16y ago
    Ah, I see now. Hiding on the "untested" branch. :-) Sorry for any confusion then, and I stand corrected.
- nilio 16y ago
  I was so excited to see just how alive this is! I was hunting around and about to start the project myself when I saw this and thought everyone else would be so excited to see that a package release was just put out a few hours ago. Great to see and I think that with some support we can really help these guys along and all really enjoy some wonderful fruit.
redial 16y ago
Does anyone know if Oracle is still developing ZFS, or just Btrfs? Has the situation changed so that Apple may include it on next iterations of OS X, or are we stuck with HSF+ forever?
- wmf 16y ago
  ZFS development continues; they recently released encryption support. Based on the rumors, it sounds like Apple is now writing their own filesystem.
  - newman314 16y ago
    Too bad. I wonder why they did not throw support after btrfs if there were licensing issues with zfs.
    - wmf 16y ago
      From Apple's perspective, Btrfs has worse licensing than ZFS.
- alwillis 16y ago
  ZFS is more appropriate for servers; I suspect we'll see some developments in the filesystem area later this year as Mac OS X 10.7 "Lion" gets closer to shipping.
  - nilio 16y ago
    ZFS is wonderful for workstations. It keeps redundant copies of blocks and has parity functionality to keep you from having silent corruption and allowing a bad drive to sneak up on you. Running a consulting business I see many hundreds of computers yearly come in with just this problem alone for hard drive failure and ZFS would make everyone's lives so much happier just on workstations alone.
zdw 16y ago
Does this support newer zfs and zpool versions? (newer version = more features) Here's a list of where other platforms are at: http://en.wikipedia.org/wiki/ZFS#Comparisons http://en.wikipedia.org/wiki/ZFS#Comparisons
ComputerGuru 16y ago
What I'm actually more interested in is what Apple is intending on using instead of ZFS. All indications are that ZFS will not be the next filesystem for OS X, yet they'll need to replace HFS soon seeing as it's the worst-performing and fewest-featured filesystem in use with this generation of operating systems. HFS+ is showing its age in multiple places, and Apple could really take advantage of some of the newer features available in modern filesystems to make a huge difference and increase the power and performance behind some of the current features (a la Time Machine), with techniques such as snapshots, transactioned reads/writes, and more.
- wtallis 16y ago
  Don't forget about the block-level deduplication and copy-on-write which would make Time Machine use much less space.
  - johnthedebs 16y ago
    I recently got Time Machine working on a network drive that was ZFS underneath (deduplication turned on) and it didn't make a very big difference. Deduplication would be nice, but I think it won't show a lot of benefit under many common workloads. Also, it can use up a ton of memory for larger drives (it would be a huge boon for SSDs, though).
    - wtallis 16y ago
      Deduplication matters most for people who edit big files. In my case, I have some virtual machines with hard drive images stored on my main filesystem. If I didn't exclude them from Time Machine, they would completely fill my backup drive in a matter of days, but with block-level deduplication, there would only be slight overhead compared with storing the same data directly on the HFS+ file system instead of in the images.
    - ComputerGuru 16y ago
      Really the biggest culprit is Mail.app - so this would depend on how big your Mail archive is.
nilio 16y ago
ZFS runs amazingly well on FreeBSD at the moment. Very, very stable and incredible. I love having to not worry about bad blocks / sectors anymore! I also love raid-z2 which allows 2 failed drives simultaneously. Just an amazing comfort. We can see this on Mac OS X with some effort!
BonoboBoner 16y ago
What I am interested in is, are there any ZFS patents that Oracle could use to sue Apple once they eventually broadly ship ZFS?
- johnthedebs 16y ago
  Apple won't be broadly shipping ZFS, which they made pretty clear when they pulled it from Snow Leopard. They haven't made an official statement, but the general consensus is that they couldn't agree on licensing terms with Sun (ZFS is licensed under the CDDL) and so they bailed on it. Oh how I wish they had worked things out. That said, Oracle is also unlikely to do them any favors.
  - yardie 16y ago
    That said, Oracle is also unlikely to do them any favors. You can always hope. It's not like Ellison and Jobs aren't BFFs. Oracle did write a version of their DB engine for OSX server for no, discernable, reason at all. I don't remember any customers asking for it particularly so this decision had to come down from way up top.
  - zdw 16y ago
    It's not CDDL - Apple has shipped dtrace for years now which is CDDL code. Most likely, ZFS was written off for various technical reasons (large RAM requirements to run well, not optimized for portable hardware, etc.) and if there was a legal issue it was probably the NetApp lawsuit against Sun regarding ZFS.
- nilio 16y ago
  I think a side community project is perfect given this potential concern (even though likelihood is unlikely). Oracle's not going to sue Apple for something like this.
fdhjrjk 16y ago
Click on our website: === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== Gold autumn, personality Mes clothing + Shoes, Travel bag that grabs an eye coat + Chao packet Free transport === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ====
joubert 16y ago
ZFS is fantastic; it was my file system when I ran Solarid on my Sun workstation (until I replaced it with Ubuntu). I did get bitten once, wiping out a whole bunch of data very unexpectedly, which I thought was a great ZFS weakness - http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg09919.html http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg0...
jmcnevin 16y ago
I'll be that guy. breath = noun breathe = verb "Breathes" New Life That is all. :)

70 points by nilio 16y ago

melvinram 16y ago

For those (like me) who are clueless on what ZFS is, the wiki page has a good overview: In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity (protection against bit rot, etc), support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. http://en.wikipedia.org/wiki/ZFS http://en.wikipedia.org/wiki/ZFS

ihodes 16y ago
ZFS is really cool. The best things that stood out out to me when I researched it a bit a while ago (though there is so much cool about it) are 1) more storage than you could ever possibly use–you won't run out. 2) you can add and remove drives on the fly. 3) you don't have drives you mount, you have pools you can add to and remove from (on the fly). 4) great redundancy/error-checking built in, in all forms. I'd love to see it on the Mac, officially…
- dlsspy 16y ago
  You can add and replace, but you cannot remove drives.
  - Andrewski 16y ago
    Sure you can. You just can't shrink the size of a pool.
- patrickgzill 16y ago
  The best feature is that you can tell it to do a "zpool scrub poolname" and it will go through the entire set of drives in that pool and check the checksums, correcting as necessary. Given that there are errors that can occur silently, ZFS can detect and correct them if they are single-bit errors.

js2 16y ago

"Breaths new life" is a bit misleading' no? From the project page: "this code project will be used to record issues and ongoing development (if any) of this extension" As well, only a single commit to the git repo since July. :(

anywho 16y ago
the forum is very active, especially alex blewitt. There are fairly infrequent updates, but it's definiatly being developed.
dlsspy 16y ago
Depends on which one you're looking at. Alex's repo has a tremendous amount of work. I've been doing little work since the initial build and installer since work picked up. I went ahead and merged mine up.
- js2 16y ago
  Ah, I see now. Hiding on the "untested" branch. :-) Sorry for any confusion then, and I stand corrected.
nilio 16y ago
I was so excited to see just how alive this is! I was hunting around and about to start the project myself when I saw this and thought everyone else would be so excited to see that a package release was just put out a few hours ago. Great to see and I think that with some support we can really help these guys along and all really enjoy some wonderful fruit.

redial 16y ago

Does anyone know if Oracle is still developing ZFS, or just Btrfs? Has the situation changed so that Apple may include it on next iterations of OS X, or are we stuck with HSF+ forever?

wmf 16y ago
ZFS development continues; they recently released encryption support. Based on the rumors, it sounds like Apple is now writing their own filesystem.
- newman314 16y ago
  Too bad. I wonder why they did not throw support after btrfs if there were licensing issues with zfs.
  - wmf 16y ago
    From Apple's perspective, Btrfs has worse licensing than ZFS.
alwillis 16y ago
ZFS is more appropriate for servers; I suspect we'll see some developments in the filesystem area later this year as Mac OS X 10.7 "Lion" gets closer to shipping.
- nilio 16y ago
  ZFS is wonderful for workstations. It keeps redundant copies of blocks and has parity functionality to keep you from having silent corruption and allowing a bad drive to sneak up on you. Running a consulting business I see many hundreds of computers yearly come in with just this problem alone for hard drive failure and ZFS would make everyone's lives so much happier just on workstations alone.

zdw 16y ago

Does this support newer zfs and zpool versions? (newer version = more features) Here's a list of where other platforms are at: http://en.wikipedia.org/wiki/ZFS#Comparisons http://en.wikipedia.org/wiki/ZFS#Comparisons

ComputerGuru 16y ago

What I'm actually more interested in is what Apple is intending on using instead of ZFS. All indications are that ZFS will not be the next filesystem for OS X, yet they'll need to replace HFS soon seeing as it's the worst-performing and fewest-featured filesystem in use with this generation of operating systems. HFS+ is showing its age in multiple places, and Apple could really take advantage of some of the newer features available in modern filesystems to make a huge difference and increase the power and performance behind some of the current features (a la Time Machine), with techniques such as snapshots, transactioned reads/writes, and more.

wtallis 16y ago
Don't forget about the block-level deduplication and copy-on-write which would make Time Machine use much less space.
- johnthedebs 16y ago
  I recently got Time Machine working on a network drive that was ZFS underneath (deduplication turned on) and it didn't make a very big difference. Deduplication would be nice, but I think it won't show a lot of benefit under many common workloads. Also, it can use up a ton of memory for larger drives (it would be a huge boon for SSDs, though).
  - wtallis 16y ago
    Deduplication matters most for people who edit big files. In my case, I have some virtual machines with hard drive images stored on my main filesystem. If I didn't exclude them from Time Machine, they would completely fill my backup drive in a matter of days, but with block-level deduplication, there would only be slight overhead compared with storing the same data directly on the HFS+ file system instead of in the images.
  - ComputerGuru 16y ago
    Really the biggest culprit is Mail.app - so this would depend on how big your Mail archive is.

nilio 16y ago

ZFS runs amazingly well on FreeBSD at the moment. Very, very stable and incredible. I love having to not worry about bad blocks / sectors anymore! I also love raid-z2 which allows 2 failed drives simultaneously. Just an amazing comfort. We can see this on Mac OS X with some effort!

BonoboBoner 16y ago

What I am interested in is, are there any ZFS patents that Oracle could use to sue Apple once they eventually broadly ship ZFS?

johnthedebs 16y ago
Apple won't be broadly shipping ZFS, which they made pretty clear when they pulled it from Snow Leopard. They haven't made an official statement, but the general consensus is that they couldn't agree on licensing terms with Sun (ZFS is licensed under the CDDL) and so they bailed on it. Oh how I wish they had worked things out. That said, Oracle is also unlikely to do them any favors.
- yardie 16y ago
  That said, Oracle is also unlikely to do them any favors. You can always hope. It's not like Ellison and Jobs aren't BFFs. Oracle did write a version of their DB engine for OSX server for no, discernable, reason at all. I don't remember any customers asking for it particularly so this decision had to come down from way up top.
- zdw 16y ago
  It's not CDDL - Apple has shipped dtrace for years now which is CDDL code. Most likely, ZFS was written off for various technical reasons (large RAM requirements to run well, not optimized for portable hardware, etc.) and if there was a legal issue it was probably the NetApp lawsuit against Sun regarding ZFS.
nilio 16y ago
I think a side community project is perfect given this potential concern (even though likelihood is unlikely). Oracle's not going to sue Apple for something like this.

fdhjrjk 16y ago

Click on our website: === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== Gold autumn, personality Mes clothing + Shoes, Travel bag that grabs an eye coat + Chao packet Free transport === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ==== === http://goo.gl/l5v0b http://goo.gl/l5v0b ====

joubert 16y ago

ZFS is fantastic; it was my file system when I ran Solarid on my Sun workstation (until I replaced it with Ubuntu). I did get bitten once, wiping out a whole bunch of data very unexpectedly, which I thought was a great ZFS weakness - http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg09919.html http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg0...

jmcnevin 16y ago

I'll be that guy. breath = noun breathe = verb "Breathes" New Life That is all. :)