4 ms·

I never really used anything in the apple ecosystem until the m1 came out. It was such an interesting device that I wound up picking it up. I was reading about

I never really used anything in the apple ecosystem until the m1 came out. It was such an interesting device that I wound up picking it up. I was reading about macos' history with filesystems a while back and apparently they were considering switching to zfs at some point but instead decided to design apfs as their own in house next gen filesystem. In my experience with it it seems to lack almost all of the features that would have made zfs interesting. One of the most obvious usecases would be for something like time machine. I have a parallels virtual machine that is hundreds of gigabytes in size and as far as I can tell that each time you modify even a single byte in that vm it will need to backup the entire file all over again. not only is this infeasible in the amount of space it would require it's also infeasible in that every time I do a backup i would need to spend the time transfering the whole thing all over again. this is also one of the biggest problems that next gen filesystems were designed to solve. is it really the case that apple's next generation filesystem doesn't support snapshots or block level replication when one of their most obvious usecases for it would be time machine backups?

EricE 4y ago
They ended up backing away from ZFS because of patents and Oracle. The irony.
_8j50 4y ago
It might be due to licensing, even Linux went with btrfs (not sure if they mainlined zfs yet).
- wfriesen 4y ago
  Not mainlined now, and probably not ever given that Linus has said "considering Oracle's litigious nature, and the questions over licensing, there's no way I can feel safe in ever doing so" https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/ https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s...
- deleted 4y ago
  [deleted]
SllX 4y ago
It’s been long enough that I can’t remember if the project was already basically dead or not going anywhere by this point, but whatever the final nail was and whether it was nailed in before or after Sun was bought, Oracle buying Sun would have just completely cremated the project no matter what state it was in.
- dottrap 4y ago
  I got to speak to a couple different Apple file system engineers many years ago over the course of many WWDCs. I think it was around the Leopard or Snow Leopard time frame (2007-2009), where we casually discussed some of the challenges Apple was having with their ZFS effort. (We didn't talk about iPhone, so either it had not been announced yet, or development was still so closed down, that it never came up as a topic of consideration.) My recollection was with their ZFS work, while it was clear to everybody that Time Machine should be a big benefactor of ZFS, they were still unhappy with the high RAM requirements (and also not thrilled about the high CPU requirements). Since laptops were then the bulk of their sales, and these machines shipped with much more constrained specs, there was a lot of uncertainty about when/if they were going to make ZFS the default file system. (Looking it up, the base 2006 Macbook had 512 MB of RAM, and shared RAM with the Intel GMA 950 GPU.) I recall there always was a secondary backdrop of concerns with the ZFS license and also the fate of Sun, but the engineers I spoke with weren't involved in those parts of the decision making process.
  - throw0101b 4y ago
    > I recall there always was a secondary backdrop of concerns with the ZFS license From one of the co-creators of ZFS: > Apple can currently just take the ZFS CDDL code and incorporate it > (like they did with DTrace), but it may be that they wanted a "private > license" from Sun (with appropriate technical support and > indemnification), and the two entities couldn't come to mutually > agreeable terms. I cannot disclose details, but that is the essence of it. * https://web.archive.org/web/20121221111757/http://mail.opensolaris.org/pipermail/zfs-discuss/2009-October/033125.html https://web.archive.org/web/20121221111757/http://mail.opens... * https://arstechnica.com/gadgets/2009/10/apple-abandons-zfs-on-mac-os-x-project-over-licensing-issues/ https://arstechnica.com/gadgets/2009/10/apple-abandons-zfs-o...
astrange 4y ago
apfs has snapshots, just not block-level snapshots.
AnonC 4y ago
> One of the most obvious usecases would be for something like time machine. I have a parallels virtual machine that is hundreds of gigabytes in size and as far as I can tell that each time you modify even a single byte in that vm it will need to backup the entire file all over again. not only is this infeasible in the amount of space it would require it's also infeasible in that every time I do a backup i would need to spend the time transfering the whole thing all over again. this is also one of the biggest problems that next gen filesystems were designed to solve. This is a likely an issue with Time Machine’s design rather than any limitation in APFS. Time Machine works with full files and uses hard links to link different versions. Unlike HFS+, APFS does support CoW (Copy on Write) and snapshots. You can verify this yourself by duplicating a file on your local drive and seeing that it doesn’t actually create one full copy of he file and occupy double the space.
- ShadowBanThis01 4y ago
  Why do Time Machine volumes inevitably become corrupt? This is a known problem but I don't remember the explanation.
  - cj 4y ago
    For me, it was forgetting the password on the external hard drive. After that, iCloud got to a point where I felt like backing up became unnecessary so I stopped TM. (I know this isn't the response you were looking for. I've also experienced corrupt TM unrelated to HD encryption, not sure why)
  - Tagbert 4y ago
    Nor is it inevitable. I have never run across a corrupt volume in TM.
    - ShadowBanThis01 4y ago
      I have, and I rely on Carbon Copy Cloner instead as a result. The reports have continued for years, right into this one: https://forums.macrumors.com/threads/how-is-this-an-acceptable-failure-mode-time-machine-backups.2379996/ https://forums.macrumors.com/threads/how-is-this-an-acceptab... I recommend surveying the extant reports on the issue. I've read several analyses that assert that corruption is indeed inevitable, and my experience was consistent with that. But hey, roll the dice if you want. I'm just providing information.
babypuncher 4y ago
I don't know anything about APFS, but I can't blame Apple for deciding to stay the hell away from Oracle
kalleboo 4y ago
It made a lot of sense for Apple to adopt ZFS back when they were shipping spinning rust hard disks and PowerMacs had room for multiple drives, but now that they're shipping soldered-in integrated flash drives with their own storage controller in the SoC, it probably makes more sense to them to put features like encryption and error correction in their storage controller rather than the file system, and then have a file system which is far more lightweight (so they can ship the same FS on a wristwatch as on a 20-core Mac). APFS does have snapshots but I don't think it has replication. Time Machine is so bad and has seen so little work, I feel like Apple really wants to drop it and move everyone to iCloud subscriptions instead. And yet they still don't have a cloud backup product, which is even stranger.
galad87 4y ago
Time Machine has continued to use HFS+ until recently, I think TM on APFS was introduced in macOS 11, and it requires to start a new backup from scratch. However, it should support block level diffing.
ribit 4y ago
APFS does have snapshots and at least the local TM backups heavily use them (it takes a fraction of a second to perform a local TM backup). TM backups to disk also seem to be organised as snapshots and have become much much faster in the recent macOS versions, so I don't think it does only copy changed blocks (but I can't be sure). What I would love for APFS to have thought is block-level deduplication. Seems like an obvious fit for an SSD-optimized COW system anyway, surprised they haven't implemented it.

EricE 4y ago

They ended up backing away from ZFS because of patents and Oracle. The irony.

_8j50 4y ago

It might be due to licensing, even Linux went with btrfs (not sure if they mainlined zfs yet).

wfriesen 4y ago
Not mainlined now, and probably not ever given that Linus has said "considering Oracle's litigious nature, and the questions over licensing, there's no way I can feel safe in ever doing so" https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/ https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s...
deleted 4y ago
[deleted]

SllX 4y ago

It’s been long enough that I can’t remember if the project was already basically dead or not going anywhere by this point, but whatever the final nail was and whether it was nailed in before or after Sun was bought, Oracle buying Sun would have just completely cremated the project no matter what state it was in.

dottrap 4y ago
I got to speak to a couple different Apple file system engineers many years ago over the course of many WWDCs. I think it was around the Leopard or Snow Leopard time frame (2007-2009), where we casually discussed some of the challenges Apple was having with their ZFS effort. (We didn't talk about iPhone, so either it had not been announced yet, or development was still so closed down, that it never came up as a topic of consideration.) My recollection was with their ZFS work, while it was clear to everybody that Time Machine should be a big benefactor of ZFS, they were still unhappy with the high RAM requirements (and also not thrilled about the high CPU requirements). Since laptops were then the bulk of their sales, and these machines shipped with much more constrained specs, there was a lot of uncertainty about when/if they were going to make ZFS the default file system. (Looking it up, the base 2006 Macbook had 512 MB of RAM, and shared RAM with the Intel GMA 950 GPU.) I recall there always was a secondary backdrop of concerns with the ZFS license and also the fate of Sun, but the engineers I spoke with weren't involved in those parts of the decision making process.
- throw0101b 4y ago
  > I recall there always was a secondary backdrop of concerns with the ZFS license From one of the co-creators of ZFS: > Apple can currently just take the ZFS CDDL code and incorporate it > (like they did with DTrace), but it may be that they wanted a "private > license" from Sun (with appropriate technical support and > indemnification), and the two entities couldn't come to mutually > agreeable terms. I cannot disclose details, but that is the essence of it. * https://web.archive.org/web/20121221111757/http://mail.opensolaris.org/pipermail/zfs-discuss/2009-October/033125.html https://web.archive.org/web/20121221111757/http://mail.opens... * https://arstechnica.com/gadgets/2009/10/apple-abandons-zfs-on-mac-os-x-project-over-licensing-issues/ https://arstechnica.com/gadgets/2009/10/apple-abandons-zfs-o...

astrange 4y ago

apfs has snapshots, just not block-level snapshots.

AnonC 4y ago

> One of the most obvious usecases would be for something like time machine. I have a parallels virtual machine that is hundreds of gigabytes in size and as far as I can tell that each time you modify even a single byte in that vm it will need to backup the entire file all over again. not only is this infeasible in the amount of space it would require it's also infeasible in that every time I do a backup i would need to spend the time transfering the whole thing all over again. this is also one of the biggest problems that next gen filesystems were designed to solve. This is a likely an issue with Time Machine’s design rather than any limitation in APFS. Time Machine works with full files and uses hard links to link different versions. Unlike HFS+, APFS does support CoW (Copy on Write) and snapshots. You can verify this yourself by duplicating a file on your local drive and seeing that it doesn’t actually create one full copy of he file and occupy double the space.

ShadowBanThis01 4y ago
Why do Time Machine volumes inevitably become corrupt? This is a known problem but I don't remember the explanation.
- cj 4y ago
  For me, it was forgetting the password on the external hard drive. After that, iCloud got to a point where I felt like backing up became unnecessary so I stopped TM. (I know this isn't the response you were looking for. I've also experienced corrupt TM unrelated to HD encryption, not sure why)
- Tagbert 4y ago
  Nor is it inevitable. I have never run across a corrupt volume in TM.
  - ShadowBanThis01 4y ago
    I have, and I rely on Carbon Copy Cloner instead as a result. The reports have continued for years, right into this one: https://forums.macrumors.com/threads/how-is-this-an-acceptable-failure-mode-time-machine-backups.2379996/ https://forums.macrumors.com/threads/how-is-this-an-acceptab... I recommend surveying the extant reports on the issue. I've read several analyses that assert that corruption is indeed inevitable, and my experience was consistent with that. But hey, roll the dice if you want. I'm just providing information.

babypuncher 4y ago

I don't know anything about APFS, but I can't blame Apple for deciding to stay the hell away from Oracle

kalleboo 4y ago

It made a lot of sense for Apple to adopt ZFS back when they were shipping spinning rust hard disks and PowerMacs had room for multiple drives, but now that they're shipping soldered-in integrated flash drives with their own storage controller in the SoC, it probably makes more sense to them to put features like encryption and error correction in their storage controller rather than the file system, and then have a file system which is far more lightweight (so they can ship the same FS on a wristwatch as on a 20-core Mac). APFS does have snapshots but I don't think it has replication. Time Machine is so bad and has seen so little work, I feel like Apple really wants to drop it and move everyone to iCloud subscriptions instead. And yet they still don't have a cloud backup product, which is even stranger.

galad87 4y ago

Time Machine has continued to use HFS+ until recently, I think TM on APFS was introduced in macOS 11, and it requires to start a new backup from scratch. However, it should support block level diffing.

ribit 4y ago

APFS does have snapshots and at least the local TM backups heavily use them (it takes a fraction of a second to perform a local TM backup). TM backups to disk also seem to be organised as snapshots and have become much much faster in the recent macOS versions, so I don't think it does only copy changed blocks (but I can't be sure). What I would love for APFS to have thought is block-level deduplication. Seems like an obvious fit for an SSD-optimized COW system anyway, surprised they haven't implemented it.