6 ms·

Z-410: How ZFS is slowly making its way to Mac OS X

138 points by thehigherlife 16y ago

lutorm 16y ago
Nice. I just started using zfs on a home linux server. We have some large Macs, too, it would be nice to have better data integrity there.
- calloc 16y ago
  Try it on FreeBSD or Solaris 11, in general I have found it to perform better on the same hardware.
  - scrod 16y ago
    After many many disk benchmarks, we eventually found that ZFS had the best performance of all the other included file systems on the latest version FreeBSD. I highly recommend the combination.
deleted 16y ago
[deleted]
dhess 16y ago
> "There's a huge chasm between using Xsan over Fibre Channel and a USB drive with Time Machine," Brady told Ars. "That middle piece is what we're looking at—users that want the convenience of a device like a Drobo, but with more reliability and [easy verifiability]." So, ultimately, this is a hardware play, then. That makes sense, because most of the features of ZFS don't add any value to a Mac with a single internal drive and at most one or two USB drives, which is how almost all Macs are used.
- ssmoot 16y ago
  Except you could do better-than-time-machine, cheap, continuous replication. You could even TimeMachine-without-additional-drives... I mean, why should my snapshot system and my backup system be one? How about letting me use TimeMachine locally, and if I plug up an additional drive, TimeMachine asks me if I want to backup to it as well? Not a game-changer exactly, but a nice-ish upgrade potentially. The biggest upgrade would honestly just be quicker/continuous replication that doesn't drag the rest of your system down.
  - bonzoesc 16y ago
    One of the big reasons for Time Machine is to make it harder for Mac owners to lose data, either through hard drives failing, or user error. Backing up to the same drive only protects against user error. Instead of displaying a checklist of things you're protected against, Time Machine is an all-or-nothing deal, which makes the interface a lot simpler. It also enables retail locations extra options to make users happier quicker: if a customer comes in pissed with a logic board problem that's the "third strike" for that computer, the support staff can simply ask if the customer has Time Machine backups, and send the customer away with a new-in-box computer without having to worry about moving drives around and migrating files in-store. Apple's vertical enough that keeping Time Machine simple saves them employee hours around the world every day.
    - ugh 16y ago
      Lion actually makes local snapshots by default but the option to turn this on and off is quite hidden and even with local snapshots enabled the huge on/off switch for Time Machine in the preferences shows that Time Machine is turned off. All other Time Machine behavior is unchanged. This is, I think, quite a clever solution to the problems you were mentioning. (This might still change before they release Lion, though.)
  - dhess 16y ago
    > You could even TimeMachine-without-additional-drives... I mean, why should my snapshot system and my backup system be one? How about letting me use TimeMachine locally, and if I plug up an additional drive, TimeMachine asks me if I want to backup to it as well? Unless something changes between now and RTM, Lion will provide that feature out of the box: http://www.9to5mac.com/54117/time-machine-can-be-used-without-an-external-hard-drive-in-lion/ http://www.9to5mac.com/54117/time-machine-can-be-used-withou... And that's about the only feature that a typical Mac user would benefit from with ZFS, without some kind of "prosumer" or "enterprise-y" external storage device.
    - ssmoot 16y ago
      I almost always use lzjb compression. It's nearly free and gives good returns for mixed data like the sort you'll find on system drives. Binaries, text, etc. Also the reliability aspect. I haven't had HFS+ go bad on my own systems, but I've seen it several times on servers. It's on about the same level as ext3fs in that arena, which is pretty bad in my book. I get the "less is more" argument and I agree generally. ZFS is so far beyond a traditional FS though.
- dlsspy 16y ago
  > most of the features of ZFS don't add any value to a Mac with a single internal drive My favorites: * Data reliability (checksumming, etc...) * Compression * Lots and lots of undo * Including undoing entire OS upgrades * Cheap and quick partitioning * Easy disk replacements I've made use of all of these things this week. It happens that the disk I replaced is part of a two-disk flat pool, but old disk out, larger new disk in and zero downtime is awesome.
  - lloeki 16y ago
    I don't recall the details of a nice article I read a few years ago mainly about RAM and ECC but as storage size grows, the probability for any one of its bits to flick "by itself" raises. Data reliability and self-correction becomes an essential part of storage. I've played a bit with ZFS myself and really enjoyed it. It's like the sum of everything we learned about filesystems and storage and previously used to shoehorn into various historical filesystems, only this time it's a clean and simple implementation.
    - tsuraan 16y ago
      I think the term you're looking for might be "adjacent track erasure", which is the idea that the magnetic fields of adjacent tracks on a hard drive interfere, so if one track is written frequently (such as a file system metadata block), the tracks that are physicially adjacent on disk may eventually have some bits flip. It's hard to find concrete numbers on whether this happens, or how often, but it's something that people are getting more concerned with on the higher-density drives that are being made now. Of course, ZFS makes the issue pretty much moot.
Derbasti 16y ago
This. I want this! Hard disks are the only part of a computer that is not replaceable. Everything that improves data integrity is improving computer usage as a whole. Improving data integrity is improving user confidence in the machine. This is something that OSes should have done years ago. All any user ever does on a machine is manipulate and work with his personal data. Absolute data integrity should be paramount to any computing technology. Alas, it is not, and we all have (hopefully) several backups. Again: Anything that improves data integrity is improving computing as a whole. I want this.
mttwrnr 16y ago
We've seen in the past that Apple is willing to adopt superior technology. They saw the potential of adopting a unix system as well as adopting an Intel chip. This allows them to provide the same user experience while providing those users with power, should they want it.
pstuart 16y ago
Now Apple just has to buy them and make it standard.
- wmf 16y ago
  I think the article explains why this isn't going to happen.
dstein 16y ago
"16 exabytes ought to be enough for anybody"
riobard 16y ago
Does ZFS do anything to optimize for SSDs?
- wmf 16y ago
  L2ARC lets you use SSD as a disk cache. I wonder how well it works with an SD card in a MBP...
  - jonhohle 16y ago
    Newer Macs use PCI-E for the SD card interface, so it could potentially work, though you'd need a very fast SD card sticking out the side of your Mac all the time.
- esteth 16y ago
  AFAIK, ZFS does copy-on-write updates, so that's a slight form of wear levelling. So yes it does, but unintentionally.
- ghaff 16y ago
  Yes. ZFS has the concept of storage pools--what used to be called hierarchical storage management. So it can deal explicitly with storage in which some members of the pool, e.g. SSDs, are faster than others. However, this is basically a server concept. It has very little to do with how MacBooks might be used.
- dprice1 16y ago
  There are three ways you can use SSDs with ZFS. First, you can use SSDs in place of rotating rust. Second, as was previously mentioned, you can use an SSD as an extension of your filesystem cache. That's called Level-2 Adaptive Replacement Cache (L2ARC). You can also use an SSD as a logging disk to accelerate writes; that's called "Logzilla", IIRC. In commercial offerings built around these features, you typically source an SSD that is either write-biased (logzilla) or read-biased, depending on what you are trying to do.
  - zdw 16y ago

138 points by thehigherlife 16y ago

Nice. I just started using zfs on a home linux server. We have some large Macs, too, it would be nice to have better data integrity there.

calloc 16y ago
Try it on FreeBSD or Solaris 11, in general I have found it to perform better on the same hardware.
- scrod 16y ago
  After many many disk benchmarks, we eventually found that ZFS had the best performance of all the other included file systems on the latest version FreeBSD. I highly recommend the combination.

deleted 16y ago

[deleted]

dhess 16y ago

> "There's a huge chasm between using Xsan over Fibre Channel and a USB drive with Time Machine," Brady told Ars. "That middle piece is what we're looking at—users that want the convenience of a device like a Drobo, but with more reliability and [easy verifiability]." So, ultimately, this is a hardware play, then. That makes sense, because most of the features of ZFS don't add any value to a Mac with a single internal drive and at most one or two USB drives, which is how almost all Macs are used.

ssmoot 16y ago
Except you could do better-than-time-machine, cheap, continuous replication. You could even TimeMachine-without-additional-drives... I mean, why should my snapshot system and my backup system be one? How about letting me use TimeMachine locally, and if I plug up an additional drive, TimeMachine asks me if I want to backup to it as well? Not a game-changer exactly, but a nice-ish upgrade potentially. The biggest upgrade would honestly just be quicker/continuous replication that doesn't drag the rest of your system down.
- bonzoesc 16y ago
  One of the big reasons for Time Machine is to make it harder for Mac owners to lose data, either through hard drives failing, or user error. Backing up to the same drive only protects against user error. Instead of displaying a checklist of things you're protected against, Time Machine is an all-or-nothing deal, which makes the interface a lot simpler. It also enables retail locations extra options to make users happier quicker: if a customer comes in pissed with a logic board problem that's the "third strike" for that computer, the support staff can simply ask if the customer has Time Machine backups, and send the customer away with a new-in-box computer without having to worry about moving drives around and migrating files in-store. Apple's vertical enough that keeping Time Machine simple saves them employee hours around the world every day.
  - ugh 16y ago
    Lion actually makes local snapshots by default but the option to turn this on and off is quite hidden and even with local snapshots enabled the huge on/off switch for Time Machine in the preferences shows that Time Machine is turned off. All other Time Machine behavior is unchanged. This is, I think, quite a clever solution to the problems you were mentioning. (This might still change before they release Lion, though.)
- dhess 16y ago
  > You could even TimeMachine-without-additional-drives... I mean, why should my snapshot system and my backup system be one? How about letting me use TimeMachine locally, and if I plug up an additional drive, TimeMachine asks me if I want to backup to it as well? Unless something changes between now and RTM, Lion will provide that feature out of the box: http://www.9to5mac.com/54117/time-machine-can-be-used-without-an-external-hard-drive-in-lion/ http://www.9to5mac.com/54117/time-machine-can-be-used-withou... And that's about the only feature that a typical Mac user would benefit from with ZFS, without some kind of "prosumer" or "enterprise-y" external storage device.
  - ssmoot 16y ago
    I almost always use lzjb compression. It's nearly free and gives good returns for mixed data like the sort you'll find on system drives. Binaries, text, etc. Also the reliability aspect. I haven't had HFS+ go bad on my own systems, but I've seen it several times on servers. It's on about the same level as ext3fs in that arena, which is pretty bad in my book. I get the "less is more" argument and I agree generally. ZFS is so far beyond a traditional FS though.
dlsspy 16y ago
> most of the features of ZFS don't add any value to a Mac with a single internal drive My favorites: * Data reliability (checksumming, etc...) * Compression * Lots and lots of undo * Including undoing entire OS upgrades * Cheap and quick partitioning * Easy disk replacements I've made use of all of these things this week. It happens that the disk I replaced is part of a two-disk flat pool, but old disk out, larger new disk in and zero downtime is awesome.
- lloeki 16y ago
  I don't recall the details of a nice article I read a few years ago mainly about RAM and ECC but as storage size grows, the probability for any one of its bits to flick "by itself" raises. Data reliability and self-correction becomes an essential part of storage. I've played a bit with ZFS myself and really enjoyed it. It's like the sum of everything we learned about filesystems and storage and previously used to shoehorn into various historical filesystems, only this time it's a clean and simple implementation.
  - tsuraan 16y ago
    I think the term you're looking for might be "adjacent track erasure", which is the idea that the magnetic fields of adjacent tracks on a hard drive interfere, so if one track is written frequently (such as a file system metadata block), the tracks that are physicially adjacent on disk may eventually have some bits flip. It's hard to find concrete numbers on whether this happens, or how often, but it's something that people are getting more concerned with on the higher-density drives that are being made now. Of course, ZFS makes the issue pretty much moot.

Derbasti 16y ago

This. I want this! Hard disks are the only part of a computer that is not replaceable. Everything that improves data integrity is improving computer usage as a whole. Improving data integrity is improving user confidence in the machine. This is something that OSes should have done years ago. All any user ever does on a machine is manipulate and work with his personal data. Absolute data integrity should be paramount to any computing technology. Alas, it is not, and we all have (hopefully) several backups. Again: Anything that improves data integrity is improving computing as a whole. I want this.

mttwrnr 16y ago

We've seen in the past that Apple is willing to adopt superior technology. They saw the potential of adopting a unix system as well as adopting an Intel chip. This allows them to provide the same user experience while providing those users with power, should they want it.

pstuart 16y ago

Now Apple just has to buy them and make it standard.

wmf 16y ago
I think the article explains why this isn't going to happen.

dstein 16y ago

"16 exabytes ought to be enough for anybody"

riobard 16y ago

Does ZFS do anything to optimize for SSDs?

wmf 16y ago
L2ARC lets you use SSD as a disk cache. I wonder how well it works with an SD card in a MBP...
- jonhohle 16y ago
  Newer Macs use PCI-E for the SD card interface, so it could potentially work, though you'd need a very fast SD card sticking out the side of your Mac all the time.
esteth 16y ago
AFAIK, ZFS does copy-on-write updates, so that's a slight form of wear levelling. So yes it does, but unintentionally.
ghaff 16y ago
Yes. ZFS has the concept of storage pools--what used to be called hierarchical storage management. So it can deal explicitly with storage in which some members of the pool, e.g. SSDs, are faster than others. However, this is basically a server concept. It has very little to do with how MacBooks might be used.
dprice1 16y ago
There are three ways you can use SSDs with ZFS. First, you can use SSDs in place of rotating rust. Second, as was previously mentioned, you can use an SSD as an extension of your filesystem cache. That's called Level-2 Adaptive Replacement Cache (L2ARC). You can also use an SSD as a logging disk to accelerate writes; that's called "Logzilla", IIRC. In commercial offerings built around these features, you typically source an SSD that is either write-biased (logzilla) or read-biased, depending on what you are trying to do.
- zdw 16y ago