9 ms·

New NTFS Read-Write Driver from Paragon Merged to Linux Kernel

176 points by beshrkayali 5y ago

azinman2 5y ago
I’d love to know once this is merged and has journaling if there are advantages to use NTFS instead of ext4 or other file systems as your main Linux FS, versus interop with windows drives.
- generalizations 5y ago
  IIRC, NTFS doesn't support permissions.
  - zamadatix 5y ago
    config NTFS3_FS_POSIX_ACL bool "NTFS POSIX Access Control Lists" depends on NTFS3_FS select FS_POSIX_ACL help POSIX Access Control Lists (ACLs) support additional access rights for users and groups beyond the standard owner/group/world scheme, and this option selects support for ACLs specifically for ntfs filesystems. NOTE: this is linux only feature. Windows will ignore these ACLs.
    - znpy 5y ago
      I'm not sure, but iirc POSIX ACL and typical Unix permission (ala chmod 0755) are two different things. Unix permissions are those you change via chmod while POSIX ACLs are those you view/change with getfacl/setfacl. NTFS probably has that because Windows used to have (still has? dunno) a full POSIX subsystem / compatibility layer.
  - deleted 5y ago
    [deleted]
  - Flimm 5y ago
    NTFS actually does support permissions, it's Windows that doesn't. NTFS also supports symbolic links and hard links.
    - nolok 5y ago
      Not contradicting just genuinely wondering, what do you mean by windows doesn't? I have different permissions setup for various users on different folders on ntfs volumes in regular windows 10 installs. Or do you mean another sort of permissions?
      - zamadatix 5y ago
        In this context they are talking about the POSIX permissions related to running Linux on the drive which NTFS supports but Windows does not use. Instead Windows uses another attribute for their security descriptor model which is also used by other non-file objects throughout the OS.
        jpgvm 5y ago
        A lot of this stuff is actually legacy from when NT hosted POSIX and Windows subsystems simultaneously on the same kernel. Sadly that stuff is long since dead but they have in part revived some of the ideas with their SQL Server for Linux port and WSL of course.
        jpgvm 5y ago
        Not sure why I'm getting downvoted here. When POSIX subsystem of the NT kernel they added support for many POSIX style features. Namely POSIX ACL support, hard links and case-sensitivity. I guess the reality of this is lost to history and folks like me that remember NT for what it was before it was relegated to just hosting Win32.
        iso8859-1 5y ago
        Does it support suid?
- Kenji 5y ago
  NTFS is shit. Ext4 is better in every single way, _especially_ performance. Ditch the Windows shit if you can.
- phendrenad2 5y ago
  Not much honestly, NTFS is known for corrupting data over a long enough timeline (so it isn't suitable for server storage).
  - dataflow 5y ago
    > NTFS is known for corrupting data over a long enough timeline Any links?
    - phendrenad2 5y ago
      Apparently it's called "bit rot": https://www.reddit.com/r/DataHoarder/comments/67h28o/bit_rot_detection_and_correction_on_windows/ https://www.reddit.com/r/DataHoarder/comments/67h28o/bit_rot...
      - dataflow 5y ago
        Bit rot has nothing to do with NTFS though? It's a general hardware degradation issue. (Where does that say it's NTFS-related?)
        deleted 5y ago
        [deleted]
        timbit42 5y ago
        There are filesystems (eg. ZFS, BtrFS) which can detect and recover from hardware degradation. NTFS cannot, therefore the problem is NTFS-related, although not limited to NTFS.
        dataflow 5y ago
        If that's what is meant, it's sure as heck misleading to single out NTFS with "known for corrupting data" as if it's somehow unique in not having error correction compared to ext4 or whatever.
        5y ago
- gaudat 5y ago
  Being able to use long file names. The "256 bytes' ought to be enough" attitude hurts in this age of emojis.
- atoav 5y ago
  I am running a dual-boot system with BTRFS on the linux side and WinBTRFS[1] to make the partitions readable from the Windows side. This works quite well (which means I never had any issue with it). [1]: https://github.com/maharmstone/btrfs https://github.com/maharmstone/btrfs
rubyist5eva 5y ago
Looking forward to testing this out with games installed in my Windows partition. Hopefully won’t need to have a dedicated partition for Linux games anymore.
- compsciphd 5y ago
  why couldn't you have used ntfs3g before then in read-only mode. or even the only ntfs in kernel code in read only mode.
  - zamadatix 5y ago
    Some games flat out won't work with read only and for the ones that do if they are still receiving regular updates it's going to be a PITA. Also a PITA when you want to add a new game. Not to mention both ntfs-3g and even the previous kernel driver weren't exactly performance marvels.
    - compsciphd 5y ago
      I'd hazard to say that games that get updates wouldn't always work in a dual boot env either. i.e. if the update stores data in the registry / game depends on registry settings.
jtvjan 5y ago
Now all that's left is for Microsoft to add EXT support to NT.
- xook 5y ago
  There are third party tools to read from Ext partitions, but unfortunately, writing to the filesystem seems to be a footgun roulette situation.
- zamadatix 5y ago
  Since they've went the way of having WSL2 physical mounts show up in \wsl$ this summer I don't think there is much push for native <pick your FS> support. https://devblogs.microsoft.com/commandline/access-linux-filesystems-in-windows-and-wsl-2/ https://devblogs.microsoft.com/commandline/access-linux-file... Would definitely be cool but Linux supporting NTFS only makes it even less necessary on top of that.
  - ecnahc515 5y ago
    I hadn’t considered how this being in the upstream kernel would make it possible to run my WSL2 on NTFS natively. That’s pretty cool. I’m sure there’s still some stuff to figure out, but still pretty awesome.
    - phkahler 5y ago
      Doesn't WSL2 already have the ability to read NTFS somehow? Isnt this just an optimization?
      - flatiron 5y ago
        It’s a network mount. Hence it’s “don’t use it cause it’s shitty” reputation.
    - zamadatix 5y ago
      Well you could do that too but you'd lose permissions between the two (NTFS supports POSIX permissions but Linux doesn't and vice versa with Windows permissions). The blog post is really about making any Linux mountable FS available to the Windows side of WSL2 without NT needing to support it. E.g. it doesn't matter if the drive is NTFS or Ext4 or BobYourUncleFS as long as the Linux kernel can mount the FS you can mount it in WSL2 and access it from Windows without NT needing to support the OS. It also already works in reverse, volumes mounted in Windows (including NTFS) can also be shared with WSL2. It's really just their integrated-into-the-OS version of VM guest filesystem sharing.
deleted 5y ago
[deleted]
- deleted 5y ago
  [deleted]
canadaduane 5y ago
Does anyone know what confluence of events or motives made this land now? I'm grateful for their work--just curious why a commercial software company decided to make its main capability available for free.
- kasabali 5y ago
  Some context: https://arstechnica.com/gadgets/2021/08/paragon-is-working-to-get-its-ntfs3-filesystem-into-the-linux-kernel/ https://arstechnica.com/gadgets/2021/08/paragon-is-working-t... So just a guess; maybe after Samsung's exfat code was merged into mainline, license and support revenue from their own exfat driver took a hit. So they decided it's in their best interest to preemptively get their own ntfs driver in mainline thus they can get more support contracts this way? PS: I noticed this time it's Tuxera who isn't happy with this development. They remembered they maintain ntfs-3g [1], and decided up the version number of ntfs-3g after 4 years [2]. So I guess the theory above makes at least a slight sense. [1] https://lore.kernel.org/lkml/alpine.DEB.2.20.2109030047330.23375@tuxera.com/ https://lore.kernel.org/lkml/alpine.DEB.2.20.2109030047330.2... [2] https://github.com/tuxera/ntfs-3g/wiki/NTFS-3G-Release-History https://github.com/tuxera/ntfs-3g/wiki/NTFS-3G-Release-Histo...
- ch_123 5y ago
  There’s an official statement here: https://www.paragon-software.com/home/ntfs3-driver-faq/ https://www.paragon-software.com/home/ntfs3-driver-faq/ Reading between the lines, I suspect that the effort of maintaining an out-of-tree file system outweighed any money to be made from selling NTFS support to Linux users, especially when ntfs-3g is probably good enough for many use cases. They have a lot of other products, and it’s possible that some of them leverage this NTFS support as well.
  - m4rtink 5y ago
    Anything out of the kernel source tree is pain in the ass for many reasons, ranging from a nuisance to impossible to use due to tainted kernel nullifying OS vendor support guarantees.
  - tgsovlerkhgsel 5y ago
jumelles 5y ago
Is this as big of a deal as it seems?
- webmobdev 5y ago
  Paragon's NTFS implementation is perceived as more mature than the alternatives. And since Windows + Linux on one system is quite common, this will make Linux more attractive for many Windows users.
phendrenad2 5y ago
Does this mean we can boot Windows and Linux from the same partition? Just point the EFI boot-loader at a different kernel image for each? Or does NTFS not support mixed NT-style and Unix-style file permissions on the same partition. Or does the Linux kernel refuse to boot from NTFS, because Linux people hate anything with a Microsoft smell?
- mgbmtl 5y ago
  I think that, worst case, you could have a /boot partition on ext4, and then / could be NTFS. Although at that point maybe it's best to have just the /home running on NTFS. I'm not too familiar with how EFI handles it, but I think that grub would need some NTFS support? No need for hate. We just get grumpy when Microsoft enforces patents or calls Linux a virus, which they haven't done in a few years ;)
  - deathanatos 5y ago
    > I'm not too familiar with how EFI handles it, but I think that grub would need some NTFS support? I use EFI, but I keep all of grub, the kernel, and the initramfs on the EFI partition. So grub wouldn't need to know anything at all about the partition for /, as it isn't loading anything from it.
- mappu 5y ago
  I think you can do this today with some other filesystems. e.g. WinBtrfs is (rightly!) very proud of being able to boot a full Windows OS.
  - phendrenad2 5y ago
    Oh wow, thanks for that, that's a cool project.
- gizmo686 5y ago
  It's not really up to the kernel. The kernel and initial ramdisk are loaded into memory by the bootloader. Once you are in the ramdisk you can do pretty much whatever you want to mount the root partition; including using a user space file system. If you wanted to, you could skip switching to the "real" root filesystem, and run your system entirely out of the ramdisk. With EFI, you can even skip having a seperate boot-loader entirely, and combine the kernel, ramdisk, and an EFI shim into a single EFI executable.
mgerdts 5y ago
And ksmbd is in Linus' tree. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e24c567b7ecff1c8b6023a10d7f78256cef742c4 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- southerntofu 5y ago
  For readers who don't understand what that is or how that's related, ksmbd appears to be a modern kernel-side implementation of the SMB (Samba) protocol which is used for sharing files across machines. I believe that's the protocol some people use to interop with Windows Active Directory for example.

176 points by beshrkayali 5y ago

I’d love to know once this is merged and has journaling if there are advantages to use NTFS instead of ext4 or other file systems as your main Linux FS, versus interop with windows drives.

generalizations 5y ago
IIRC, NTFS doesn't support permissions.
- zamadatix 5y ago
  config NTFS3_FS_POSIX_ACL bool "NTFS POSIX Access Control Lists" depends on NTFS3_FS select FS_POSIX_ACL help POSIX Access Control Lists (ACLs) support additional access rights for users and groups beyond the standard owner/group/world scheme, and this option selects support for ACLs specifically for ntfs filesystems. NOTE: this is linux only feature. Windows will ignore these ACLs.
  - znpy 5y ago
    I'm not sure, but iirc POSIX ACL and typical Unix permission (ala chmod 0755) are two different things. Unix permissions are those you change via chmod while POSIX ACLs are those you view/change with getfacl/setfacl. NTFS probably has that because Windows used to have (still has? dunno) a full POSIX subsystem / compatibility layer.
- deleted 5y ago
  [deleted]
- Flimm 5y ago
  NTFS actually does support permissions, it's Windows that doesn't. NTFS also supports symbolic links and hard links.
  - nolok 5y ago
    Not contradicting just genuinely wondering, what do you mean by windows doesn't? I have different permissions setup for various users on different folders on ntfs volumes in regular windows 10 installs. Or do you mean another sort of permissions?
    - zamadatix 5y ago
      In this context they are talking about the POSIX permissions related to running Linux on the drive which NTFS supports but Windows does not use. Instead Windows uses another attribute for their security descriptor model which is also used by other non-file objects throughout the OS.
      - jpgvm 5y ago
        A lot of this stuff is actually legacy from when NT hosted POSIX and Windows subsystems simultaneously on the same kernel. Sadly that stuff is long since dead but they have in part revived some of the ideas with their SQL Server for Linux port and WSL of course.
        jpgvm 5y ago
        Not sure why I'm getting downvoted here. When POSIX subsystem of the NT kernel they added support for many POSIX style features. Namely POSIX ACL support, hard links and case-sensitivity. I guess the reality of this is lost to history and folks like me that remember NT for what it was before it was relegated to just hosting Win32.
        iso8859-1 5y ago
        Does it support suid?
Kenji 5y ago
NTFS is shit. Ext4 is better in every single way, _especially_ performance. Ditch the Windows shit if you can.
phendrenad2 5y ago
Not much honestly, NTFS is known for corrupting data over a long enough timeline (so it isn't suitable for server storage).
- dataflow 5y ago
  > NTFS is known for corrupting data over a long enough timeline Any links?
  - phendrenad2 5y ago
    Apparently it's called "bit rot": https://www.reddit.com/r/DataHoarder/comments/67h28o/bit_rot_detection_and_correction_on_windows/ https://www.reddit.com/r/DataHoarder/comments/67h28o/bit_rot...
    - dataflow 5y ago
      Bit rot has nothing to do with NTFS though? It's a general hardware degradation issue. (Where does that say it's NTFS-related?)
      - deleted 5y ago
        [deleted]
      - timbit42 5y ago
        There are filesystems (eg. ZFS, BtrFS) which can detect and recover from hardware degradation. NTFS cannot, therefore the problem is NTFS-related, although not limited to NTFS.
        dataflow 5y ago
        If that's what is meant, it's sure as heck misleading to single out NTFS with "known for corrupting data" as if it's somehow unique in not having error correction compared to ext4 or whatever.
        5y ago
gaudat 5y ago
Being able to use long file names. The "256 bytes' ought to be enough" attitude hurts in this age of emojis.
atoav 5y ago
I am running a dual-boot system with BTRFS on the linux side and WinBTRFS[1] to make the partitions readable from the Windows side. This works quite well (which means I never had any issue with it). [1]: https://github.com/maharmstone/btrfs https://github.com/maharmstone/btrfs

rubyist5eva 5y ago

Looking forward to testing this out with games installed in my Windows partition. Hopefully won’t need to have a dedicated partition for Linux games anymore.

compsciphd 5y ago
why couldn't you have used ntfs3g before then in read-only mode. or even the only ntfs in kernel code in read only mode.
- zamadatix 5y ago
  Some games flat out won't work with read only and for the ones that do if they are still receiving regular updates it's going to be a PITA. Also a PITA when you want to add a new game. Not to mention both ntfs-3g and even the previous kernel driver weren't exactly performance marvels.
  - compsciphd 5y ago
    I'd hazard to say that games that get updates wouldn't always work in a dual boot env either. i.e. if the update stores data in the registry / game depends on registry settings.

jtvjan 5y ago

Now all that's left is for Microsoft to add EXT support to NT.

xook 5y ago
There are third party tools to read from Ext partitions, but unfortunately, writing to the filesystem seems to be a footgun roulette situation.
zamadatix 5y ago
Since they've went the way of having WSL2 physical mounts show up in \wsl$ this summer I don't think there is much push for native <pick your FS> support. https://devblogs.microsoft.com/commandline/access-linux-filesystems-in-windows-and-wsl-2/ https://devblogs.microsoft.com/commandline/access-linux-file... Would definitely be cool but Linux supporting NTFS only makes it even less necessary on top of that.
- ecnahc515 5y ago
  I hadn’t considered how this being in the upstream kernel would make it possible to run my WSL2 on NTFS natively. That’s pretty cool. I’m sure there’s still some stuff to figure out, but still pretty awesome.
  - phkahler 5y ago
    Doesn't WSL2 already have the ability to read NTFS somehow? Isnt this just an optimization?
    - flatiron 5y ago
      It’s a network mount. Hence it’s “don’t use it cause it’s shitty” reputation.
  - zamadatix 5y ago
    Well you could do that too but you'd lose permissions between the two (NTFS supports POSIX permissions but Linux doesn't and vice versa with Windows permissions). The blog post is really about making any Linux mountable FS available to the Windows side of WSL2 without NT needing to support it. E.g. it doesn't matter if the drive is NTFS or Ext4 or BobYourUncleFS as long as the Linux kernel can mount the FS you can mount it in WSL2 and access it from Windows without NT needing to support the OS. It also already works in reverse, volumes mounted in Windows (including NTFS) can also be shared with WSL2. It's really just their integrated-into-the-OS version of VM guest filesystem sharing.

deleted 5y ago

[deleted]

deleted 5y ago
[deleted]

canadaduane 5y ago

Does anyone know what confluence of events or motives made this land now? I'm grateful for their work--just curious why a commercial software company decided to make its main capability available for free.

kasabali 5y ago
Some context: https://arstechnica.com/gadgets/2021/08/paragon-is-working-to-get-its-ntfs3-filesystem-into-the-linux-kernel/ https://arstechnica.com/gadgets/2021/08/paragon-is-working-t... So just a guess; maybe after Samsung's exfat code was merged into mainline, license and support revenue from their own exfat driver took a hit. So they decided it's in their best interest to preemptively get their own ntfs driver in mainline thus they can get more support contracts this way? PS: I noticed this time it's Tuxera who isn't happy with this development. They remembered they maintain ntfs-3g [1], and decided up the version number of ntfs-3g after 4 years [2]. So I guess the theory above makes at least a slight sense. [1] https://lore.kernel.org/lkml/alpine.DEB.2.20.2109030047330.23375@tuxera.com/ https://lore.kernel.org/lkml/alpine.DEB.2.20.2109030047330.2... [2] https://github.com/tuxera/ntfs-3g/wiki/NTFS-3G-Release-History https://github.com/tuxera/ntfs-3g/wiki/NTFS-3G-Release-Histo...
ch_123 5y ago
There’s an official statement here: https://www.paragon-software.com/home/ntfs3-driver-faq/ https://www.paragon-software.com/home/ntfs3-driver-faq/ Reading between the lines, I suspect that the effort of maintaining an out-of-tree file system outweighed any money to be made from selling NTFS support to Linux users, especially when ntfs-3g is probably good enough for many use cases. They have a lot of other products, and it’s possible that some of them leverage this NTFS support as well.
- m4rtink 5y ago
  Anything out of the kernel source tree is pain in the ass for many reasons, ranging from a nuisance to impossible to use due to tainted kernel nullifying OS vendor support guarantees.
- tgsovlerkhgsel 5y ago

jumelles 5y ago

Is this as big of a deal as it seems?

webmobdev 5y ago
Paragon's NTFS implementation is perceived as more mature than the alternatives. And since Windows + Linux on one system is quite common, this will make Linux more attractive for many Windows users.

phendrenad2 5y ago

Does this mean we can boot Windows and Linux from the same partition? Just point the EFI boot-loader at a different kernel image for each? Or does NTFS not support mixed NT-style and Unix-style file permissions on the same partition. Or does the Linux kernel refuse to boot from NTFS, because Linux people hate anything with a Microsoft smell?

mgbmtl 5y ago
I think that, worst case, you could have a /boot partition on ext4, and then / could be NTFS. Although at that point maybe it's best to have just the /home running on NTFS. I'm not too familiar with how EFI handles it, but I think that grub would need some NTFS support? No need for hate. We just get grumpy when Microsoft enforces patents or calls Linux a virus, which they haven't done in a few years ;)
- deathanatos 5y ago
  > I'm not too familiar with how EFI handles it, but I think that grub would need some NTFS support? I use EFI, but I keep all of grub, the kernel, and the initramfs on the EFI partition. So grub wouldn't need to know anything at all about the partition for /, as it isn't loading anything from it.
mappu 5y ago
I think you can do this today with some other filesystems. e.g. WinBtrfs is (rightly!) very proud of being able to boot a full Windows OS.
- phendrenad2 5y ago
  Oh wow, thanks for that, that's a cool project.
gizmo686 5y ago
It's not really up to the kernel. The kernel and initial ramdisk are loaded into memory by the bootloader. Once you are in the ramdisk you can do pretty much whatever you want to mount the root partition; including using a user space file system. If you wanted to, you could skip switching to the "real" root filesystem, and run your system entirely out of the ramdisk. With EFI, you can even skip having a seperate boot-loader entirely, and combine the kernel, ramdisk, and an EFI shim into a single EFI executable.

mgerdts 5y ago

And ksmbd is in Linus' tree. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e24c567b7ecff1c8b6023a10d7f78256cef742c4 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

southerntofu 5y ago
For readers who don't understand what that is or how that's related, ksmbd appears to be a modern kernel-side implementation of the SMB (Samba) protocol which is used for sharing files across machines. I believe that's the protocol some people use to interop with Windows Active Directory for example.