10 ms·

Nice. But I would like it better if the effort was to support ext4, BtrFS, NTFS and other popular filesystems from the Linux and Windows world...

gjsman-1000 1y ago
ext4 and Btrfs are only well supported on Linux; they are not universal standards. NTFS was only supported well on Windows until recently; but extensions like NTFS Encryption (BitLocker) are still Windows only. Mac still does not let you write to an NTFS volume. APFS and HFS+ are obviously Apple file systems. FreeBSD does not support ext4 or Btrfs well; but instead prefers UFS2 or ZFS despite also being an open-source Unix-inspired OS. The world runs on proprietary or non-universal file systems with CDFS (ISO 9660), FAT, and exFAT being the sole exceptions.
- nemomarx 1y ago
  if Mac also supported Linux file formats they would get closer to universal, though? we can't get to that world without these individual steps. creating a new non universal one is backing away from it
  - gjsman-1000 1y ago
    Even FreeBSD can't be bothered to do a good job supporting Linux file systems. They're basically proprietary siloed file systems like the rest, even if code is available. Linux, meanwhile, can't be bothered to support either of FreeBSD's file systems officially. UFS2 is hardly patented anymore, but Linux doesn't care beyond read-only support. Being open-source, and even being in a popular open-source project, does not make a standard, or even imply inferiority to those who do not implement it. https://xkcd.com/927/ https://xkcd.com/927/
    - bombcar 1y ago
      There might be some license issues, but the dirty secret is filesystem portability isn't terribly important, and for the users for whom it is - ExFAT and friends are usually good enough.
      - johnfernow 1y ago
        exFAT is widely used but it not being journaled has led to so many thousands (if not more) people losing tons of data, many of which wouldn't have lost so much data had they used a journaled filesystem (or even one with redundant file tables.) If you need to connect a portable drive to machines of different OS's, there is no safe filesystem that supports read and write on both Windows and MacOS. Alternatively, cloud storage works until the files are larger than the space you have left on Drive/Dropbox/OneDrive/etc., and local network sharing (on certain networks at least) is more complicated than what the average user is willing to put up with. In practice, many use USB flash drives or external HDDs/SSDs with exFAT. Yeah, people should have more than one backup, but we know in the real world that's not what many do. That requires them spending more time (e.g. configuring local network sharing or setting up an old machine lying around to be a NAS) or money (more on cloud storage.) In practice, having a cross-platform, journaled filesystem would lead to a lot less data loss. Aside from exFAT, the only alternative with native cross-platform R/W capability is FAT32, but while it has a redundant file allocation table (unlike exFAT), it has a max file size of 4GB, which limits its usefulness for many workflows.
  - atonse 1y ago
    Does this even matter, if you can copy/mount files over a network? What are potential use cases where you'd want to support those additional file systems? External drives?
    - cortesoft 1y ago
      I just ran into a use case yesterday. I wanted to copy some files from either my Mac or my Windows machine onto the MicroSD card for my SteamDeck, which is ext4. I wanted to just plug in the card and copy files, but couldn’t.
      - atonse 1y ago
        Interesting, ok. I don't even want to say "Use NTFS, doesn't everything support it?" because I'm not even sure that's the best solution for an SD card. Maybe with their new FSKit API [1], someone can build a compatibility layer for it. 1: https://developer.apple.com/documentation/fskit?language=objc https://developer.apple.com/documentation/fskit?language=obj...
        cortesoft 1y ago
        The best solution is to add support for EXT4 to mac and windows.
- bombcar 1y ago
  Is there a single filesystem in the world (besides "simple" ones like FAT) that both has an open standard AND is licensed in a "usable" codebase (MIT or other "non-copyleft" license)?
  - dataflow 1y ago
    I think UDF was meant to be this, though I don't think it supports everything you'd want. Does NTFS have licensing issues though?
    - arccy 1y ago
      NTFS has serious performance issues, nobody should use it.
      - dataflow 1y ago
        AFAIK that's an incorrect meme that just won't die. The performance issues you're thinking of have nothing to do with the filesystem itself, but with the I/O subsystem in Windows more generally. If you have evidence otherwise please share.
        dist-epoch 1y ago
        Microsoft itself admits there are performance issues with NTFS, which is one reason they created ReFS. For example ReFS removed the MFT which caused various problems.
        dataflow 1y ago
        I'm on my phone and this is a longer discussion than I can have here, but the performance problems they're thinking of and the ones people usually rant about on these forums are not the same ones (or same magnitude). Before being so confident it's NTFS itself that's the issue, try ReFS (and FAT32 for that matter) and tell me if you see the performance problems you've encountered actually improve a lot. And then narrow down the cause. You might be surprised how much of it had to do e.g. with filter drivers or other things. And don't forget you're still testing one particular implementation on one OS, which doesn't say anything about a different one for the same filesystem on a different OS.
- Shekelphile 1y ago
  [flagged]
  - gjsman-1000 1y ago
    A. You're flagged for graphic, inappropriate, and triggering language. B. Oracle sued Google for a decade over whether a minor number of APIs were copyrighted in Java; do you think Apple's eager to embrace their technology regardless of license? Heck, even Ubuntu wasn't willing to make that bet. C. Guess how much official licenses for those fancy file systems cost. D. ZFS, and other fancy file systems, are not known for low RAM and CPU usage. In a server, this does not matter; but it's pretty important for anything on a battery.
- mystifyingpoi 1y ago
  > NTFS was only supported well on Windows until recently I remember using ntfs-3g without any issues on my first Linux laptop 15 years ago. So that's not really "recently".
  - gjsman-1000 1y ago
    I intentionally excluded unofficial or third party software, as almost anything is supported if you’re brave enough. The quality of said drivers also wildly varies. Until 4 years ago, nothing was good enough for the upstream Linux kernel.
    - mystifyingpoi 1y ago
      I mean, yeah, you could say that. Something being in the kernel is a good benchmark of quality. But IMO open-source is different. For instance, Terraform had no stable release from 2014 till 2021, that didn't stop enterprises from using it on scale.
mrpippy 1y ago
Apple recently made FSKit a supported/documented API, which should allow third-parties to add support for other filesystems: https://developer.apple.com/documentation/fskit?language=objc https://developer.apple.com/documentation/fskit?language=obj...
- duskwuff 1y ago
  Is it actually supported and usable now? I seem to recall it spending a lot of time in a "half-documented and not actually available" state.
- whywhywhywhy 1y ago
  3rd party supporting a file system would be one of the last things on a list of all software I’d ever want a 3rd party writing instead of the OS maker. Nightmare to evaluate the options, pure stress testing the options, difficult to know if it didn’t mess something up.
  - deleted 1y ago
    [deleted]
  - Defletter 1y ago
    Question, who maintains NTFS support for Linux? Microsoft? The kernel? The distros? Genuinely asking.
    - LambdaComplex 1y ago
      The current ntfs3 driver is developed by a company called Paragon, and it's been part of the kernel since 5.15 or so. I was going to add some additional comments about this, but then I found that Paragon's website has an FAQ that covers everything I was going to say (and more): https://www.paragon-software.com/us/home/ntfs3-driver-faq/ https://www.paragon-software.com/us/home/ntfs3-driver-faq/
    - deleted 1y ago
      [deleted]
    - 1y ago
jraph 1y ago
This article describes a new disk image format (on which a filesystem can be put, APFS in the article), not a filesystem, or did I misunderstand? edit: added the word "image", which I apparently forgot to type. Mentioning the edition because otherwise it would make an answer to this comment difficult to understand.
- crazygringo 1y ago
  This is a new disk image format. Not even a disk format. For virtualizing a disk. On which you can put a filesystem, yes. There seems to be an extraordinary amount of confusion in many of the comments, I don't know why.
  - hulitu 1y ago
    > For virtualizing a disk. We really don't have enough disk formats in this space. Is this bringing something new ? Or is just polluting the namespace.
    - coldtea 1y ago
      >Is this bringing something new? If only there was a whole post available so one could find out...
      - Dylan16807 1y ago
        The post only benchmarks against UDIF. Whether this brings something new is a very good question that is not answered by the post. How complex is the format? Did they do anything clever? Did they engage in NIH? For all I know they switched the AES mode and made the blocks bigger and that's the entirety of the performance boost.
  - sitkack 1y ago
    Just reading the title, I was like, I hope it is an update to encrypted sparse bundles, and it is. https://developer.apple.com/documentation/virtualization/vzdiskimagestoragedeviceattachment/ https://developer.apple.com/documentation/virtualization/vzd...
deleted 1y ago
[deleted]
neuroelectron 1y ago
Can't abstract away your property that way.
xp84 1y ago
Does anybody else think that it would make sense for Apple and Microsoft to just get in a room and horse trade a few things like this, if they cared about user experience? Cross-license both APFS and NTFS, and share any internal documentation under NDA so that external drives can use modern formats with safety features like journaling without locking users in. Oh wait, I just answered my question.
- dwaite 1y ago
  I suspect there wouldn't be an agreement on a minimum set of features for a modern filesystem, even just for external disks, even if you limited it to flash storage devices to avoid all the complexities of spinning platter latency. After all, there's no such agreement on Linux either - we just have all the Linux vendor options available.
  - xp84 1y ago
    Let me clarify, I don't expect two vendors like that to merge filesystem specs. I just think they should have first-class support for reading and writing on each others' default (NTFS and APFS) filesystems because the alternative is that a user who has a hard drive full of their important documents can't switch between Mac and PC without buying 100% more storage with which to do a complicated filesystem-migration exercise -- and let's be honest, only an absolute nerd (like us) would even understand what that is. Others would just plug in the NTFS disk to the Mac, see the popup saying "unreadable," shrug and give up (or worst case, format it not understanding that means erasing). This is the kind of thing a reasonable government would just tell them they had to do by virtue of being a fully locked-in duopoly, just like they should tell Apple and Google that users should be able to choose to install an alternat app store.
- hypercube33 1y ago
  This is a disk image format so why not VHD? It seems like it's open enough and supports what a virtual disk needs what do we gain with yet another file that's a disk type?
deleted 1y ago
[deleted]

gjsman-1000 1y ago

ext4 and Btrfs are only well supported on Linux; they are not universal standards. NTFS was only supported well on Windows until recently; but extensions like NTFS Encryption (BitLocker) are still Windows only. Mac still does not let you write to an NTFS volume. APFS and HFS+ are obviously Apple file systems. FreeBSD does not support ext4 or Btrfs well; but instead prefers UFS2 or ZFS despite also being an open-source Unix-inspired OS. The world runs on proprietary or non-universal file systems with CDFS (ISO 9660), FAT, and exFAT being the sole exceptions.

nemomarx 1y ago
if Mac also supported Linux file formats they would get closer to universal, though? we can't get to that world without these individual steps. creating a new non universal one is backing away from it
- gjsman-1000 1y ago
  Even FreeBSD can't be bothered to do a good job supporting Linux file systems. They're basically proprietary siloed file systems like the rest, even if code is available. Linux, meanwhile, can't be bothered to support either of FreeBSD's file systems officially. UFS2 is hardly patented anymore, but Linux doesn't care beyond read-only support. Being open-source, and even being in a popular open-source project, does not make a standard, or even imply inferiority to those who do not implement it. https://xkcd.com/927/ https://xkcd.com/927/
  - bombcar 1y ago
    There might be some license issues, but the dirty secret is filesystem portability isn't terribly important, and for the users for whom it is - ExFAT and friends are usually good enough.
    - johnfernow 1y ago
      exFAT is widely used but it not being journaled has led to so many thousands (if not more) people losing tons of data, many of which wouldn't have lost so much data had they used a journaled filesystem (or even one with redundant file tables.) If you need to connect a portable drive to machines of different OS's, there is no safe filesystem that supports read and write on both Windows and MacOS. Alternatively, cloud storage works until the files are larger than the space you have left on Drive/Dropbox/OneDrive/etc., and local network sharing (on certain networks at least) is more complicated than what the average user is willing to put up with. In practice, many use USB flash drives or external HDDs/SSDs with exFAT. Yeah, people should have more than one backup, but we know in the real world that's not what many do. That requires them spending more time (e.g. configuring local network sharing or setting up an old machine lying around to be a NAS) or money (more on cloud storage.) In practice, having a cross-platform, journaled filesystem would lead to a lot less data loss. Aside from exFAT, the only alternative with native cross-platform R/W capability is FAT32, but while it has a redundant file allocation table (unlike exFAT), it has a max file size of 4GB, which limits its usefulness for many workflows.
- atonse 1y ago
  Does this even matter, if you can copy/mount files over a network? What are potential use cases where you'd want to support those additional file systems? External drives?
  - cortesoft 1y ago
    I just ran into a use case yesterday. I wanted to copy some files from either my Mac or my Windows machine onto the MicroSD card for my SteamDeck, which is ext4. I wanted to just plug in the card and copy files, but couldn’t.
    - atonse 1y ago
      Interesting, ok. I don't even want to say "Use NTFS, doesn't everything support it?" because I'm not even sure that's the best solution for an SD card. Maybe with their new FSKit API [1], someone can build a compatibility layer for it. 1: https://developer.apple.com/documentation/fskit?language=objc https://developer.apple.com/documentation/fskit?language=obj...
      - cortesoft 1y ago
        The best solution is to add support for EXT4 to mac and windows.
bombcar 1y ago
Is there a single filesystem in the world (besides "simple" ones like FAT) that both has an open standard AND is licensed in a "usable" codebase (MIT or other "non-copyleft" license)?
- dataflow 1y ago
  I think UDF was meant to be this, though I don't think it supports everything you'd want. Does NTFS have licensing issues though?
  - arccy 1y ago
    NTFS has serious performance issues, nobody should use it.
    - dataflow 1y ago
      AFAIK that's an incorrect meme that just won't die. The performance issues you're thinking of have nothing to do with the filesystem itself, but with the I/O subsystem in Windows more generally. If you have evidence otherwise please share.
      - dist-epoch 1y ago
        Microsoft itself admits there are performance issues with NTFS, which is one reason they created ReFS. For example ReFS removed the MFT which caused various problems.
        dataflow 1y ago
        I'm on my phone and this is a longer discussion than I can have here, but the performance problems they're thinking of and the ones people usually rant about on these forums are not the same ones (or same magnitude). Before being so confident it's NTFS itself that's the issue, try ReFS (and FAT32 for that matter) and tell me if you see the performance problems you've encountered actually improve a lot. And then narrow down the cause. You might be surprised how much of it had to do e.g. with filter drivers or other things. And don't forget you're still testing one particular implementation on one OS, which doesn't say anything about a different one for the same filesystem on a different OS.
Shekelphile 1y ago
[flagged]
- gjsman-1000 1y ago
  A. You're flagged for graphic, inappropriate, and triggering language. B. Oracle sued Google for a decade over whether a minor number of APIs were copyrighted in Java; do you think Apple's eager to embrace their technology regardless of license? Heck, even Ubuntu wasn't willing to make that bet. C. Guess how much official licenses for those fancy file systems cost. D. ZFS, and other fancy file systems, are not known for low RAM and CPU usage. In a server, this does not matter; but it's pretty important for anything on a battery.
mystifyingpoi 1y ago
> NTFS was only supported well on Windows until recently I remember using ntfs-3g without any issues on my first Linux laptop 15 years ago. So that's not really "recently".
- gjsman-1000 1y ago
  I intentionally excluded unofficial or third party software, as almost anything is supported if you’re brave enough. The quality of said drivers also wildly varies. Until 4 years ago, nothing was good enough for the upstream Linux kernel.
  - mystifyingpoi 1y ago
    I mean, yeah, you could say that. Something being in the kernel is a good benchmark of quality. But IMO open-source is different. For instance, Terraform had no stable release from 2014 till 2021, that didn't stop enterprises from using it on scale.

mrpippy 1y ago

Apple recently made FSKit a supported/documented API, which should allow third-parties to add support for other filesystems: https://developer.apple.com/documentation/fskit?language=objc https://developer.apple.com/documentation/fskit?language=obj...

duskwuff 1y ago
Is it actually supported and usable now? I seem to recall it spending a lot of time in a "half-documented and not actually available" state.
whywhywhywhy 1y ago
3rd party supporting a file system would be one of the last things on a list of all software I’d ever want a 3rd party writing instead of the OS maker. Nightmare to evaluate the options, pure stress testing the options, difficult to know if it didn’t mess something up.
- deleted 1y ago
  [deleted]
- Defletter 1y ago
  Question, who maintains NTFS support for Linux? Microsoft? The kernel? The distros? Genuinely asking.
  - LambdaComplex 1y ago
    The current ntfs3 driver is developed by a company called Paragon, and it's been part of the kernel since 5.15 or so. I was going to add some additional comments about this, but then I found that Paragon's website has an FAQ that covers everything I was going to say (and more): https://www.paragon-software.com/us/home/ntfs3-driver-faq/ https://www.paragon-software.com/us/home/ntfs3-driver-faq/
  - deleted 1y ago
    [deleted]
  - 1y ago

jraph 1y ago

This article describes a new disk image format (on which a filesystem can be put, APFS in the article), not a filesystem, or did I misunderstand? edit: added the word "image", which I apparently forgot to type. Mentioning the edition because otherwise it would make an answer to this comment difficult to understand.

crazygringo 1y ago
This is a new disk image format. Not even a disk format. For virtualizing a disk. On which you can put a filesystem, yes. There seems to be an extraordinary amount of confusion in many of the comments, I don't know why.
- hulitu 1y ago
  > For virtualizing a disk. We really don't have enough disk formats in this space. Is this bringing something new ? Or is just polluting the namespace.
  - coldtea 1y ago
    >Is this bringing something new? If only there was a whole post available so one could find out...
    - Dylan16807 1y ago
      The post only benchmarks against UDIF. Whether this brings something new is a very good question that is not answered by the post. How complex is the format? Did they do anything clever? Did they engage in NIH? For all I know they switched the AES mode and made the blocks bigger and that's the entirety of the performance boost.
- sitkack 1y ago
  Just reading the title, I was like, I hope it is an update to encrypted sparse bundles, and it is. https://developer.apple.com/documentation/virtualization/vzdiskimagestoragedeviceattachment/ https://developer.apple.com/documentation/virtualization/vzd...

deleted 1y ago

[deleted]

neuroelectron 1y ago

Can't abstract away your property that way.

xp84 1y ago

Does anybody else think that it would make sense for Apple and Microsoft to just get in a room and horse trade a few things like this, if they cared about user experience? Cross-license both APFS and NTFS, and share any internal documentation under NDA so that external drives can use modern formats with safety features like journaling without locking users in. Oh wait, I just answered my question.

dwaite 1y ago
I suspect there wouldn't be an agreement on a minimum set of features for a modern filesystem, even just for external disks, even if you limited it to flash storage devices to avoid all the complexities of spinning platter latency. After all, there's no such agreement on Linux either - we just have all the Linux vendor options available.
- xp84 1y ago
  Let me clarify, I don't expect two vendors like that to merge filesystem specs. I just think they should have first-class support for reading and writing on each others' default (NTFS and APFS) filesystems because the alternative is that a user who has a hard drive full of their important documents can't switch between Mac and PC without buying 100% more storage with which to do a complicated filesystem-migration exercise -- and let's be honest, only an absolute nerd (like us) would even understand what that is. Others would just plug in the NTFS disk to the Mac, see the popup saying "unreadable," shrug and give up (or worst case, format it not understanding that means erasing). This is the kind of thing a reasonable government would just tell them they had to do by virtue of being a fully locked-in duopoly, just like they should tell Apple and Google that users should be able to choose to install an alternat app store.
hypercube33 1y ago
This is a disk image format so why not VHD? It seems like it's open enough and supports what a virtual disk needs what do we gain with yet another file that's a disk type?

deleted 1y ago

[deleted]