3 ms·

I've been running ZFS in prod since 2008 (on Solaris). Bear in mind that was not long after the "silent data corruption" problem that bit Joyent, but more quot

by rodgerd 10y ago

I've been running ZFS in prod since 2008 (on Solaris). Bear in mind that was not long after the "silent data corruption" problem that bit Joyent, but more quotodien problems such as: * no reserved space on the disk means you can fill it up and then not be able to delete files or extend it because there was no space for COW operations. Sun's answer? Delete the filesystem, recreate and restore from backup. * zpool management is a pain in the arse generally, and the zpool/zfs distinction is pretty clearly designed on the assumption you'll have sysadmins paying a lot of attention to the low lever FS and pool management. Imagine Steve Jobs being told "your disk filled up and now you need to reformat it". Apple didn't reject ZFS because (whatever some embittered Sun blogger says) of NIH. They rejected it because it wasn't fit for purpose as a consumer desktop filesystem.

ScottBurson 10y ago
There's also the problem that you can't grow a raidz vdev by adding disks. I can just imagine Apple support trying to explain that one.
- lmm 10y ago
  In Solaris ZFS you can now AIUI - it's just that hasn't made it into the open-source version.
- xenophonf 10y ago
  That's true, but you can grow a pool by progressively replacing smaller disks with larger disks.
hvidgaard 10y ago
But surely, it would have been a simpler task to modify ZFS to support these things. I mean, the full disk issue is relatively straight forward to fix. And I have a hard time imaging that other issues would be harder than writing your own from scratch.
- rodgerd 10y ago
  > But surely, it would have been a simpler task to modify ZFS to support these things. The first time I encountered the disk full problem was 2009. There were emails from Sun engineers saying "this is a dumb situation, we should have a reserved block of storage to avoid this" from 2006 and nothing had been done. Why would anyone in Apple believe if was going to get better if Sun's response to paying customers was "reformat and rebuild" for at least three years? There was no way of fixing the limitations around vdevs, like not being able to grow existing arrays, not mixing block sizes without a big perfomance hit, and so on, without a huge overhaul of some of the fundamentals. ZFS was, like a lot of Sun technologies, built on assumptions from a world where machines are scarce, have a high ratio of admins:machines, and many users per machines. It's excellent at meeting its design goals, but there is no way I can imagine Apple shipping it when they'd be telling people, "want to grow your Mac Pro's array? Rebuild it. Want to reclaim space on a full system? Rebuild it. Want avoid dropping off a performance cliff? Never drop below 20% free space." (As an aside, I've been told by Sun engineers around the same 2008-2009 era that dependency resolution in packagme managers for distros proved Linux was a shit toy for morons because "real" OSes assumed skilled engineers who would be inspecting patches package-by-package so they knew exactly what would be on the system. Even recently, with Solaris 11 shipping with a package manager, I've had Sun veterans complain that "Solaris is dead" because it's "too much like Linux". Sun appears to have fostered, at least in areas I've dealt with, a culture that's completely out-of-step with how computing works this century. I doubt it would have taken too many of those sorts of conversations to convince people at Apple that Sun just didn't get Apple's "just works" vision.)
  - hvidgaard 10y ago
    I know all too well how details can prevent otherwise simple solutions. But for the full disk system, a simple solution is to not allocate all the available space, surely it cannot be hard to reserve a little bit such that you will never encounder the full volume issue. You mention it shortly, and I belive that ZFS was designed for a world where growing existing arrays, reaching 80+% utilization, or using different block sizes all is things that never happen anyway. ZFS could do with a rewrite to allow this, but it would not be ZFS compatible anymore. But I will not deny that would you describe could be a telltale sign of a development team Apple simply cannot work with. But nothing prevent Apple from forking OpenZFS and work on it in house (perhaps apart from patent issues).

by rodgerd 10y ago

ScottBurson 10y ago

There's also the problem that you can't grow a raidz vdev by adding disks. I can just imagine Apple support trying to explain that one.

lmm 10y ago
In Solaris ZFS you can now AIUI - it's just that hasn't made it into the open-source version.
xenophonf 10y ago
That's true, but you can grow a pool by progressively replacing smaller disks with larger disks.

hvidgaard 10y ago

But surely, it would have been a simpler task to modify ZFS to support these things. I mean, the full disk issue is relatively straight forward to fix. And I have a hard time imaging that other issues would be harder than writing your own from scratch.

rodgerd 10y ago
> But surely, it would have been a simpler task to modify ZFS to support these things. The first time I encountered the disk full problem was 2009. There were emails from Sun engineers saying "this is a dumb situation, we should have a reserved block of storage to avoid this" from 2006 and nothing had been done. Why would anyone in Apple believe if was going to get better if Sun's response to paying customers was "reformat and rebuild" for at least three years? There was no way of fixing the limitations around vdevs, like not being able to grow existing arrays, not mixing block sizes without a big perfomance hit, and so on, without a huge overhaul of some of the fundamentals. ZFS was, like a lot of Sun technologies, built on assumptions from a world where machines are scarce, have a high ratio of admins:machines, and many users per machines. It's excellent at meeting its design goals, but there is no way I can imagine Apple shipping it when they'd be telling people, "want to grow your Mac Pro's array? Rebuild it. Want to reclaim space on a full system? Rebuild it. Want avoid dropping off a performance cliff? Never drop below 20% free space." (As an aside, I've been told by Sun engineers around the same 2008-2009 era that dependency resolution in packagme managers for distros proved Linux was a shit toy for morons because "real" OSes assumed skilled engineers who would be inspecting patches package-by-package so they knew exactly what would be on the system. Even recently, with Solaris 11 shipping with a package manager, I've had Sun veterans complain that "Solaris is dead" because it's "too much like Linux". Sun appears to have fostered, at least in areas I've dealt with, a culture that's completely out-of-step with how computing works this century. I doubt it would have taken too many of those sorts of conversations to convince people at Apple that Sun just didn't get Apple's "just works" vision.)
- hvidgaard 10y ago
  I know all too well how details can prevent otherwise simple solutions. But for the full disk system, a simple solution is to not allocate all the available space, surely it cannot be hard to reserve a little bit such that you will never encounder the full volume issue. You mention it shortly, and I belive that ZFS was designed for a world where growing existing arrays, reaching 80+% utilization, or using different block sizes all is things that never happen anyway. ZFS could do with a rewrite to allow this, but it would not be ZFS compatible anymore. But I will not deny that would you describe could be a telltale sign of a development team Apple simply cannot work with. But nothing prevent Apple from forking OpenZFS and work on it in house (perhaps apart from patent issues).