22 ms·

Paragon submits 27k-line NTFS driver to Linux kernel

203 points by kiyanwang 6y ago

mariuolo 6y ago
I don't think it's completely fair. Don't look a gift horse in the mouth, especially when you need one.
- Pick-A-Hill2019 6y ago
  I think their (the kernel maintainers’) position is that a single patch of 27000 lines of diffs is a bit of a nightmare to do code review on. I’m not sure if you took a look at the patch file (available at https://dl.paragon-software.com/ntfs3/ntfs3.patch https://dl.paragon-software.com/ntfs3/ntfs3.patch ). I think their point is ‘man, how are we going to divide this up amongst the maintainers? Who gets to check which function or call?’ Paragons response (more or less will fix in v2) https://lore.kernel.org/linux-fsdevel/a8fa5b2b31b349f2858306af01269eda@paragon-software.com/ https://lore.kernel.org/linux-fsdevel/a8fa5b2b31b349f2858306...
  - tyingq 6y ago
    Using language like "monstrosity" though, doesn't help.
    - znpy 6y ago
      I guess that's not you that have to do the review.
      - tyingq 6y ago
        Heh. I've certainly been handed stuff I didn't like. It's always been easier to work through if I avoided anything that came off as emotional or insulting.
    - boomboomsubban 6y ago
      The word has several meanings, and I took it to mean "frighteningly large" rather than any of the more negative ones. Perhaps not the best choice, but not overtly rude.
    - hodgesrm 6y ago
      "Monstrosity" seems like a fair word, and to the point as well. The follow-up messages showed good will on the part of the kernel maintainers.
    - sorryd4d 6y ago
      Sorry, dad. We’ll all try really hard to ignore our individual biology to satisfy the most sensitive sensibilities. Cause that expectation is not manipulative at all.
      - dang 6y ago
        Could you please stop posting flamebait and unsubstantive comments to HN? Also, could you please stop creating accounts for every few comments you post? We ban accounts that do that. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=community%20identity%20by:dang https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
  - dboreham 6y ago
    If it's working code (assume it is) then requiring it to be turd-polished for the purposes of code review is absurd. Whoever is asking for that is either disingenuous or has no real experience as a software developer.
    - hanselot 6y ago
      You would merge 27k of "working" code into one of your main projects based entirely on good faith? Now lets ask rather, would you merge 27k of "working" code into something that runs 90% of the planet?
- znpy 6y ago
  On one hand yes, on the other hand if you want your patch to be included it's your job to make it reviewable and to make it pass all the various checks. After all, we all lived long enough without their software, we can live without it a bit longer. Yes it's a gift, but it's also code that has to be maintained and updated along with the kernel, so it's not really 100% a gift. If you then consider that they might keep on selling a proprietary version of the code (which - don't get me wrong - is 100% legit and fair) they might also get basically free labour: they could rebase onto the latest public gpl version, they might get notes of various issues and bugs... Quite literally, it's free labour.
  - hinkley 6y ago
    “Free as in puppy.”
    - DuskStar 6y ago
      This is a wonderful description of "free" to go along with "free as in beer" and "free as in speech", thanks!
    - qchris 6y ago
      It turns out someone (could be you?) wrote a Medium article about open-source software using the "free as in puppy" metaphor, although I don't think that they really used in the same way you are here with regards to a corporation using the open-source community to functionally receive free labor. I'm definitely going to add it to my mental list... [1] https://medium.com/swlh/free-as-in-puppy-5b7eb1bf3908 https://medium.com/swlh/free-as-in-puppy-5b7eb1bf3908
      - hinkley 6y ago
        Wasn’t me. Gifts bearing obligations come in a lot of shapes and sizes, I just always found the puppy metaphor very compelling and so I like to use it. (As a mental exercise sometime, go to a pet store and figure out how much a “$12 hamster” costs once you get everything you need to set up and maintain a habitat.)
    - 6y ago
- rhn_mk1 6y ago
  Taking up patches to code is also taking up vulnerabilities and bugs they carry. Even assuming good intentions, a gift of code is a gift with a burden attached, and should pass deep scrutiny before getting accepted. Sad as it may sound, free code doesn't come for free.
- mason55 6y ago
  This is like giving someone a litter of puppies as a gift and then walking away. Hooray, free puppies. Now you just have to care for them for the next 10 years.
  - setr 6y ago
    Paragon stated their intent to maintain and support the code in their initial email, and added themselves to the maintainer files. There's no drama happening here. Paragon guys are trying to give it "properly", and linux guys want it "properly", and the only thing happening is defining "proper" in this context
    - aronpye 6y ago
      Intent, promises and reality aren’t always aligned. Especially when given a slap dash of tens of thousands of lines of code that didn’t meet kernel contribution guidelines.
      - setr 6y ago
        Thats true... but nothing has happened yet. No one has yet failed to hold up their end. All parties seem to want this to work -- it certainly isn't a dead drop.
  - eirini1 6y ago
    how is that a good comparison, when you're given puppies you are being made responsible for living creatures, If you're given a bunch of code you can literally just drop it.
  - mixmastamyk 6y ago
    Someone else mentioned their commitment. Additionally, the driver is mature, as is the FS.
- guenthert 6y ago
  Working on someone else's code is no fun. I do it, when I get paid properly, but can't say I enjoy it. So I understand that not everyone is enthusiastic about that gift. And who needs it? There are alternatives to read or write the occasional file (e.g. if you happen to have forgotten the Admin's password) of a NT box, but I can't say that I missed the ability to create new files in a NTFS. And why now? Surely those who actually needed that functionality, needed it years ago and meanwhile found some other solution. Perhaps those who actually need it still volunteer to ready that driver for inclusion into the kernel or sponsor someone who can?
  - formerly_proven 6y ago
    NTFS-3g can create files and directories etc. on NTFS (albeit it doesn't do journaling, that's why it requires you to have a clean journal). Having a proper kernel driver is mostly about performance, not features. The current built-in kernel driver for NTFS is completely read-only though.
    - wtallis 6y ago
      > The current built-in kernel driver for NTFS is completely read-only though. Not quite: > CONFIG_NTFS_RW: > This enables the partial, but safe, write support in the NTFS driver. > The only supported operation is overwriting existing files, without changing the file length. No file or directory creation, deletion or renaming is possible. Note only non-resident files can be written to you may find that some very small files (<500 bytes or so) cannot be written to.
- krisoft 6y ago
  > "Don't look a gift horse in the mouth" Interesting you choose that metaphor. I can think of a particular gift horse where you would have seen Greek soldiers if you have looked into its mouth. :)
- toyg 6y ago
  > especially when you need one. It's very debatable that Linux "needs" more support for a proprietary 25-year-old filesystem that is, in many ways, obsolete.
DHowett 6y ago
LKML post: https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d74@paragon-software.com/ https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d7... Earlier discussion: https://news.ycombinator.com/item?id=24170001 https://news.ycombinator.com/item?id=24170001
- cozzyd 6y ago
  Thanks, maybe it was the expectations set by the tone of The Register article, but the mailing list discussion seemed mostly very reasonable...
londons_explore 6y ago
As long as it builds as a module, uses only the standard filesystem apis and doesn't have code changes outside that, code review is far less important IMO...
- huhtenberg 6y ago
  At the risk of stating the obvious, this is an exceptionally short-sighted approach. Committing poorly understood code is not that different from using binary blobs.
- lumost 6y ago
  The linux kernel is successful partly because by and large what's in the kernel works and doesn't generally break on new revisions. Accepting a new driver which may in fact not work, breaks in ambiguous ways, interacts with other components poorly, or otherwise generates headaches for the kernel could be more trouble than it's worth. Ultimately this driver will eventually need to be modified by others, and ensuing it doesn't get a reputation as a nightmare right out of the gate is also worthwhile.
- aronpye 6y ago
  So you’d willingly allow tens of thousands of lines of code which you don’t understand or really know what it does into a kernel which is used by literally billions of devices?
- cratermoon 6y ago
  > uses only the standard filesystem apis and doesn't have code changes outside that That's literally why having to review 27K lines of code is a pain. You're not seriously going to trust, sight unseen, that there's nothing in the code that slips in a backdoor or catastrophic data-loss bug, are you?
- segmondy 6y ago
  I wish you weren't getting downvoted, but what you said is absolutely correct. If it's a module and separate from core, who cares. If it's modifying core kernel or mixed up with kernel space code, then nope.
piscisaureus 6y ago
I understand that reviewing 27k LoC is daunting and probably not very fun. But unlike most patches that draw a similar response it's not a narrowly useful patch that mostly serves the submitter. Proper NTFS support benefits a large proportion of Linux users (the jab from the article that there are more advanced file systems out there seems out of place; there are no signs that windows is about to switch its default FS to something else). Additionally this code has been used in production for years now (e.g. my 2015 router runs the closed source version of this driver in order to support NTFS formatted external drives) so most likely a lot of quality issues have already been found and addressed. So I feel it's a bit unreasonable to respond with so much negativity to this contribution.
- AnotherGoodName 6y ago
  It's also no big deal from either side. Paragon sent in the patch and it's appreciated. There's a few problems to get this in. Reviewers noted the issues and what would need to be done to get this through. The process to get this in is happening. Split your diff! and Fix your makefile! have to be one of the most benign and common pieces of diff feedback i've seen. I feel that you could make a media story about any submission to the Linux kernel based on there being comments in the review process.
  - piscisaureus 6y ago
    Admittedly I didn't actually read the mailing list discussion. It's entirely possible that The Register made up a big drama where there was none.
    - echlebek 6y ago
      There wasn't much drama in the mailing list discussion as I read it. Mostly comments asking "can you make this easier for us to review?".
    - mehrdadn 6y ago
      Not sure if this counts as drama per Linux kernel mailing list standards: https://lore.kernel.org/linux-fsdevel/2911ac5cd20b46e397be506268718d74@paragon-software.com/t/#m34fc84d93a7009caf533ff400f7c763da4cab366 https://lore.kernel.org/linux-fsdevel/2911ac5cd20b46e397be50... > So how exactly do you expect someone to review this monstrosity ?
sam42 6y ago
In the meantime v2 of the patch has already been submitted, addressing some of the points mentioned in the article: https://lore.kernel.org/linux-fsdevel/904d985365a34f0787a4511435417ab3@paragon-software.com/ https://lore.kernel.org/linux-fsdevel/904d985365a34f0787a451...
mauvehaus 6y ago
Possibly a dumb question, but what are the plusses/minuses of something like this as compared to NTFS-3G? https://en.wikipedia.org/wiki/NTFS-3G https://en.wikipedia.org/wiki/NTFS-3G
- formerly_proven 6y ago
  Mostly better performance / less CPU usage. NTFS-3G is typically CPU-limited even for sequential access, which isn't exactly a great spot for a FS driver to be in.
- solarkraft 6y ago
  NTFS-3G is a userspace (FUSE) driver, which usually have worse performance than kernel drivers.
- dTal 6y ago
  You'll be able to install Linux on NTFS root now!
  - nix23 6y ago
    Nice idea, and with Wine we could have all the beautiful Windows AV-Software :-)
  - mehrdadn 6y ago
    I wish Linux on NTFS could share metadata with WSL1. That way you could boot it off the same WSL1 files you could access in Windows.
nippoo 6y ago
This is such an odd article. Perhaps Paragon isn't being entirely altruistic with this move but TR are being quite scathing of someone's work submitting a kernel driver with no direct financial reward - and no real praise for, hopefully, fixing one of the biggest out-of-the-box gripes that Windows / Linux desktop dual-boot environments have. It's no small wonder people often complain about the hardships of writing and maintaining open-source software!
- shakna 6y ago
  I don't get where TheRegister is getting the drama. The thread doesn't seem that scathing to me. [0] It doesn't build, but the person who pointed it out also supplied a diff to make it happen. It also fails a few tests, but Paragon are more than happy to see if they can make it a bit more compliant. UBSan finds a few potential bugs, but again, Paragon are more than happy to fix the problems. There's some style guide suggestions, which Paragon seem to immediately take on board: > The patch will be splitted in v2 file-wise. Wasn't clear initially which way will be more convenient to review. [0] https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d74@paragon-software.com/ https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d7...
  - Delk 6y ago
    Yeah, The Register can often be witty and irreverent, and I wouldn't complain about that, but that kind of wit only works if there's an actual point behind it. Jumping on the apparently popular bandwagon of squeezing out "drama" from the Linux kernel mailing list isn't adding anything of value. It also seems a bit ignorant to call the submission "half-baked" just because it needed to be worked on and because someone pointed that out (also) in a slightly irreverent way. All of those points you bring up about the feedback they got seem like just business as usual for a larger merge of code to a carefully developed FOSS project.
  - cortesoft 6y ago
    Yeah, this seems like one of those 'dramas' that incites bystanders more than the actual participants.
- 6y ago
Tade0 6y ago
I solemnly swear never to complain again about the size of PRs given to me for review. It's going to take months to throughly assess the quality of this.
Havoc 6y ago
Ouch. That looks really useful, but at the same time possibly their first kernel submission perhaps? Hope I’m wrong but I think they’ll be fighting initial bad impression for a while despite good intention
cactus2093 6y ago
It's strange to me that there's no file system with basic features like journaling and support for files larger than a couple of GB, that is supported across all major desktop OSes (MacOS, Windows, Linux, and FreeBSD). All platforms support the same i/o standards like USB or DisplayPort, why did filesystems never make the cut to become a cross-system standard? Imagine if you could have a backup drive (with reasonable modern data protections) that you could just plug into different systems and save all your files to. Isn't it odd that such a simple thing isn't possible? I guess network attached storage has gotten pretty accessible at this point so there's no need for it?
- quarantine 6y ago
  There's a ZFS driver for Linux, macOS, and Windows.
  - wtallis 6y ago
    In all three cases you list, it's a third-party module providing support, rather than it being a standard feature of the OS that can be expected to be generally available on more than 1% of the install base.
  - hoistbypetard 6y ago
    ZFS development moves so fast that it is common for my (FreeBSD-based) FreeNAS box to warn me when I upgrade my OS that certain actions will make it incompatible with the prior version of FreeNAS. That is fine and appropriate for a drive that will be connected to the system for the foreseeable future. That kind of compatibility concern makes me squeamish about using ZFS for a drive that I want to share between different systems. If it's easy to make it incompatible between two releases for the same system, that smells like a waiting nightmare trying to keep it compatible between Linux, FreeBSD and Windows.
    - DenseComet 6y ago
      Yeah it should stabilize in the next couple of months with the release of OpenZFS 2.0 as that release is supposed to signify the unification of ZFS on Linux and FreeBSD. ZFS on FreeBSD is being rebased onto ZFS on Linux. Theres also been some talk on adding MacOS zfs support OpenZFS but thats still up in the air.
beervirus 6y ago
27k is 27 kB. This is 27 kLOC.
prirun 6y ago
It's just a single data point, but I tried Paragon's ext4 driver (the paid version) for Mac many years ago. It seemed to work, but when the drive was connected back to Linux, there were all kinds of fsck errors. Immediately deleted it.
- apfsx 6y ago
  This has also been my exact experience. I used Paragon's NTFS driver (paid) for Mac for my external SSD. After using, when plugging into Windows it would always find "errors" and recommend to Scan and Fix them.
  - Wowfunhappy 6y ago
    Fwiw, I had the same problem with Paragon but have had a lot more success with Tuxera’s driver, if you’re still looking for a solution.
adamretter 6y ago
I hope the it's better than their ext2 drivers for macOS. I have had nothing but problems and data loss and their support is the worst.
- jug 6y ago
  Same on Windows regarding data loss. Terrible and I’ll never use their ext2 drivers again.
deleted 6y ago
[deleted]
aaron695 6y ago
> "It looks as though that with NTFS being surpassed by other more advanced file-systems, Is this remotly true? Will my grandmother not be using NTFS in 10 years?
- wtallis 6y ago
  Microsoft has been trying to come up with a replacement for NTFS for a very long time. They've had mixed success with trying to extend NTFS with more advanced features like the now-deprecated TxF. There's little doubt that even its creators see NTFS as something of a dead-end. Whether your grandmother is still using it in 10 years depends primarily on whether Microsoft can get its act together to pick and ship a replacement.
  - aaron695 6y ago
    > Whether your grandmother is still using it in 10 years depends primarily on whether Microsoft can get its act together to pick and ship a replacement. So yes she will I guess. Then it's vital Linux gets NTFS working well. My partner is not going to use Linux things if every time they try and transfer the 8k 3D holographic photos of our CRISPR'ed dog learning to spell to my grandmother it doesn't work on her Holovision. True story, last month I just lost about 1 in 10 of my media files on my Linux Share to NTFS issues. So now I run the box on Windows.
    - wtallis 6y ago
      > True story, last month I just lost about 1 in 10 of my media files on my Linux Share to NTFS issues. So now I run the box on Windows. Were you seriously running a Linux-based NAS with NTFS as the underlying filesystem, or did you mean something very different? I can't imagine why anyone would ever think their choice of disk filesystem on a server—hiding behind a network filesystem—should be influenced by what disk filesystems are supported by client devices.
      - aaron695 6y ago
        Given my original question about my grandmother has been downvoted it says it all about the judginess of Linux users ;) My setup was - Windows and Ubuntu dual boot gaming PC with a NTFS 4G external hard disk, Windows network share. Default boot was to Ubuntu. Torrents running off wireless laptop to the network share. Amazon Fire stick with Kodi wireless running to network share.
PhoenixRobo 6y ago
Does it need to be included in the Linux Kernel? I understand that NTFS is 27 years old and mostly to support Microsoft file systems. Can't the driver be an external download?
- EE84M3i 6y ago
  In-tree drivers are the standard for Linux kernel modules.
bArray 6y ago
27k lines isn't that crazy. I've merged larger patches, it just takes a while. This is a very negative view of an open source contribution.
- wtallis 6y ago
  > I've merged larger patches, In what kind of context? The Linux kernel? Filesystem code is pretty tricky to begin with, and prone to very subtle bugs with very not-subtle consequences. And this isn't greenfield development of a new filesystem, but an implementation that needs to remain highly compatible with Microsoft's version. This FS driver has to be maintained to track changes to two operating systems. So this 27kloc can reasonably be expected to encompass a lot more complexity than your average 27kloc, and it requires a lot more review effort than something like 27kloc of GPU driver register definitions.
  - bArray 6y ago
    > In what kind of context? The Linux kernel? Not the Linux kernel, but a large embedded system. > Filesystem code is pretty tricky to begin with, and prone > to very subtle bugs with very not-subtle consequences. This code has been running in the wild for quite a while now, it has had a trial by fire. And there's no way around testing, subtle ext4 bugs still crop up despite the maturity of the filesystem. > And this isn't greenfield development of a new filesystem, > but an implementation that needs to remain highly > compatible with Microsoft's version. No amount of code review will stop Microsoft from adapting their version. Also, I doubt Microsoft themselves will change too much about the filesystem given the compatibility they themselves have to maintain with cold storage NTFS drives. > This FS driver has to be maintained to track changes to > two operating systems. You make it sound as if Microsoft have a hand in any of this. Also, have you seen the state of the current NTFS driver? It's a bit flakey (no disrespect to the maintainers).
    - wtallis 6y ago
      > No amount of code review will stop Microsoft from adapting their version. Way to miss the point. Code review for the kernel isn't just about verifying that the code currently works. It's also about making sure the code is maintainable. Microsoft is relevant here because their actions will increase the maintenance burden of any Linux NTFS driver. Kernel developers rightly need to be concerned about how difficult it will be to extend the NTFS driver to handle new NTFS features that Microsoft introduces.
userbinator 6y ago
I wonder if the reason for the huge line count is overly verbose code, or if it's just the inherent complexity of NTFS. For contrast, I wrote a FAT32 filesystem driver (read/write) for an embedded system a long time ago, and it was less than 1K lines --- of Asm.
- The_Colonel 6y ago
  NTFS is way more complicated (and feature rich) filesystem than (rather barebones) FAT32.
r41nbowdash 6y ago
200loc/h, it's 4-5 weeks of work for a single person

203 points by kiyanwang 6y ago

mariuolo 6y ago

I don't think it's completely fair. Don't look a gift horse in the mouth, especially when you need one.

Pick-A-Hill2019 6y ago
I think their (the kernel maintainers’) position is that a single patch of 27000 lines of diffs is a bit of a nightmare to do code review on. I’m not sure if you took a look at the patch file (available at https://dl.paragon-software.com/ntfs3/ntfs3.patch https://dl.paragon-software.com/ntfs3/ntfs3.patch ). I think their point is ‘man, how are we going to divide this up amongst the maintainers? Who gets to check which function or call?’ Paragons response (more or less will fix in v2) https://lore.kernel.org/linux-fsdevel/a8fa5b2b31b349f2858306af01269eda@paragon-software.com/ https://lore.kernel.org/linux-fsdevel/a8fa5b2b31b349f2858306...
- tyingq 6y ago
  Using language like "monstrosity" though, doesn't help.
  - znpy 6y ago
    I guess that's not you that have to do the review.
    - tyingq 6y ago
      Heh. I've certainly been handed stuff I didn't like. It's always been easier to work through if I avoided anything that came off as emotional or insulting.
  - boomboomsubban 6y ago
    The word has several meanings, and I took it to mean "frighteningly large" rather than any of the more negative ones. Perhaps not the best choice, but not overtly rude.
  - hodgesrm 6y ago
    "Monstrosity" seems like a fair word, and to the point as well. The follow-up messages showed good will on the part of the kernel maintainers.
  - sorryd4d 6y ago
    Sorry, dad. We’ll all try really hard to ignore our individual biology to satisfy the most sensitive sensibilities. Cause that expectation is not manipulative at all.
    - dang 6y ago
      Could you please stop posting flamebait and unsubstantive comments to HN? Also, could you please stop creating accounts for every few comments you post? We ban accounts that do that. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=community%20identity%20by:dang https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
- dboreham 6y ago
  If it's working code (assume it is) then requiring it to be turd-polished for the purposes of code review is absurd. Whoever is asking for that is either disingenuous or has no real experience as a software developer.
  - hanselot 6y ago
    You would merge 27k of "working" code into one of your main projects based entirely on good faith? Now lets ask rather, would you merge 27k of "working" code into something that runs 90% of the planet?
znpy 6y ago
On one hand yes, on the other hand if you want your patch to be included it's your job to make it reviewable and to make it pass all the various checks. After all, we all lived long enough without their software, we can live without it a bit longer. Yes it's a gift, but it's also code that has to be maintained and updated along with the kernel, so it's not really 100% a gift. If you then consider that they might keep on selling a proprietary version of the code (which - don't get me wrong - is 100% legit and fair) they might also get basically free labour: they could rebase onto the latest public gpl version, they might get notes of various issues and bugs... Quite literally, it's free labour.
- hinkley 6y ago
  “Free as in puppy.”
  - DuskStar 6y ago
    This is a wonderful description of "free" to go along with "free as in beer" and "free as in speech", thanks!
  - qchris 6y ago
    It turns out someone (could be you?) wrote a Medium article about open-source software using the "free as in puppy" metaphor, although I don't think that they really used in the same way you are here with regards to a corporation using the open-source community to functionally receive free labor. I'm definitely going to add it to my mental list... [1] https://medium.com/swlh/free-as-in-puppy-5b7eb1bf3908 https://medium.com/swlh/free-as-in-puppy-5b7eb1bf3908
    - hinkley 6y ago
      Wasn’t me. Gifts bearing obligations come in a lot of shapes and sizes, I just always found the puppy metaphor very compelling and so I like to use it. (As a mental exercise sometime, go to a pet store and figure out how much a “$12 hamster” costs once you get everything you need to set up and maintain a habitat.)
  - 6y ago
rhn_mk1 6y ago
Taking up patches to code is also taking up vulnerabilities and bugs they carry. Even assuming good intentions, a gift of code is a gift with a burden attached, and should pass deep scrutiny before getting accepted. Sad as it may sound, free code doesn't come for free.
mason55 6y ago
This is like giving someone a litter of puppies as a gift and then walking away. Hooray, free puppies. Now you just have to care for them for the next 10 years.
- setr 6y ago
  Paragon stated their intent to maintain and support the code in their initial email, and added themselves to the maintainer files. There's no drama happening here. Paragon guys are trying to give it "properly", and linux guys want it "properly", and the only thing happening is defining "proper" in this context
  - aronpye 6y ago
    Intent, promises and reality aren’t always aligned. Especially when given a slap dash of tens of thousands of lines of code that didn’t meet kernel contribution guidelines.
    - setr 6y ago
      Thats true... but nothing has happened yet. No one has yet failed to hold up their end. All parties seem to want this to work -- it certainly isn't a dead drop.
- eirini1 6y ago
  how is that a good comparison, when you're given puppies you are being made responsible for living creatures, If you're given a bunch of code you can literally just drop it.
- mixmastamyk 6y ago
  Someone else mentioned their commitment. Additionally, the driver is mature, as is the FS.
guenthert 6y ago
Working on someone else's code is no fun. I do it, when I get paid properly, but can't say I enjoy it. So I understand that not everyone is enthusiastic about that gift. And who needs it? There are alternatives to read or write the occasional file (e.g. if you happen to have forgotten the Admin's password) of a NT box, but I can't say that I missed the ability to create new files in a NTFS. And why now? Surely those who actually needed that functionality, needed it years ago and meanwhile found some other solution. Perhaps those who actually need it still volunteer to ready that driver for inclusion into the kernel or sponsor someone who can?
- formerly_proven 6y ago
  NTFS-3g can create files and directories etc. on NTFS (albeit it doesn't do journaling, that's why it requires you to have a clean journal). Having a proper kernel driver is mostly about performance, not features. The current built-in kernel driver for NTFS is completely read-only though.
  - wtallis 6y ago
    > The current built-in kernel driver for NTFS is completely read-only though. Not quite: > CONFIG_NTFS_RW: > This enables the partial, but safe, write support in the NTFS driver. > The only supported operation is overwriting existing files, without changing the file length. No file or directory creation, deletion or renaming is possible. Note only non-resident files can be written to you may find that some very small files (<500 bytes or so) cannot be written to.
krisoft 6y ago
> "Don't look a gift horse in the mouth" Interesting you choose that metaphor. I can think of a particular gift horse where you would have seen Greek soldiers if you have looked into its mouth. :)
toyg 6y ago
> especially when you need one. It's very debatable that Linux "needs" more support for a proprietary 25-year-old filesystem that is, in many ways, obsolete.

DHowett 6y ago

LKML post: https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d74@paragon-software.com/ https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d7... Earlier discussion: https://news.ycombinator.com/item?id=24170001 https://news.ycombinator.com/item?id=24170001

cozzyd 6y ago
Thanks, maybe it was the expectations set by the tone of The Register article, but the mailing list discussion seemed mostly very reasonable...

londons_explore 6y ago

As long as it builds as a module, uses only the standard filesystem apis and doesn't have code changes outside that, code review is far less important IMO...

huhtenberg 6y ago
At the risk of stating the obvious, this is an exceptionally short-sighted approach. Committing poorly understood code is not that different from using binary blobs.
lumost 6y ago
The linux kernel is successful partly because by and large what's in the kernel works and doesn't generally break on new revisions. Accepting a new driver which may in fact not work, breaks in ambiguous ways, interacts with other components poorly, or otherwise generates headaches for the kernel could be more trouble than it's worth. Ultimately this driver will eventually need to be modified by others, and ensuing it doesn't get a reputation as a nightmare right out of the gate is also worthwhile.
aronpye 6y ago
So you’d willingly allow tens of thousands of lines of code which you don’t understand or really know what it does into a kernel which is used by literally billions of devices?
cratermoon 6y ago
> uses only the standard filesystem apis and doesn't have code changes outside that That's literally why having to review 27K lines of code is a pain. You're not seriously going to trust, sight unseen, that there's nothing in the code that slips in a backdoor or catastrophic data-loss bug, are you?
segmondy 6y ago
I wish you weren't getting downvoted, but what you said is absolutely correct. If it's a module and separate from core, who cares. If it's modifying core kernel or mixed up with kernel space code, then nope.

piscisaureus 6y ago

I understand that reviewing 27k LoC is daunting and probably not very fun. But unlike most patches that draw a similar response it's not a narrowly useful patch that mostly serves the submitter. Proper NTFS support benefits a large proportion of Linux users (the jab from the article that there are more advanced file systems out there seems out of place; there are no signs that windows is about to switch its default FS to something else). Additionally this code has been used in production for years now (e.g. my 2015 router runs the closed source version of this driver in order to support NTFS formatted external drives) so most likely a lot of quality issues have already been found and addressed. So I feel it's a bit unreasonable to respond with so much negativity to this contribution.

AnotherGoodName 6y ago
It's also no big deal from either side. Paragon sent in the patch and it's appreciated. There's a few problems to get this in. Reviewers noted the issues and what would need to be done to get this through. The process to get this in is happening. Split your diff! and Fix your makefile! have to be one of the most benign and common pieces of diff feedback i've seen. I feel that you could make a media story about any submission to the Linux kernel based on there being comments in the review process.
- piscisaureus 6y ago
  Admittedly I didn't actually read the mailing list discussion. It's entirely possible that The Register made up a big drama where there was none.
  - echlebek 6y ago
    There wasn't much drama in the mailing list discussion as I read it. Mostly comments asking "can you make this easier for us to review?".
  - mehrdadn 6y ago
    Not sure if this counts as drama per Linux kernel mailing list standards: https://lore.kernel.org/linux-fsdevel/2911ac5cd20b46e397be506268718d74@paragon-software.com/t/#m34fc84d93a7009caf533ff400f7c763da4cab366 https://lore.kernel.org/linux-fsdevel/2911ac5cd20b46e397be50... > So how exactly do you expect someone to review this monstrosity ?

sam42 6y ago

In the meantime v2 of the patch has already been submitted, addressing some of the points mentioned in the article: https://lore.kernel.org/linux-fsdevel/904d985365a34f0787a4511435417ab3@paragon-software.com/ https://lore.kernel.org/linux-fsdevel/904d985365a34f0787a451...

mauvehaus 6y ago

Possibly a dumb question, but what are the plusses/minuses of something like this as compared to NTFS-3G? https://en.wikipedia.org/wiki/NTFS-3G https://en.wikipedia.org/wiki/NTFS-3G

formerly_proven 6y ago
Mostly better performance / less CPU usage. NTFS-3G is typically CPU-limited even for sequential access, which isn't exactly a great spot for a FS driver to be in.
solarkraft 6y ago
NTFS-3G is a userspace (FUSE) driver, which usually have worse performance than kernel drivers.
dTal 6y ago
You'll be able to install Linux on NTFS root now!
- nix23 6y ago
  Nice idea, and with Wine we could have all the beautiful Windows AV-Software :-)
- mehrdadn 6y ago
  I wish Linux on NTFS could share metadata with WSL1. That way you could boot it off the same WSL1 files you could access in Windows.

nippoo 6y ago

This is such an odd article. Perhaps Paragon isn't being entirely altruistic with this move but TR are being quite scathing of someone's work submitting a kernel driver with no direct financial reward - and no real praise for, hopefully, fixing one of the biggest out-of-the-box gripes that Windows / Linux desktop dual-boot environments have. It's no small wonder people often complain about the hardships of writing and maintaining open-source software!

shakna 6y ago
I don't get where TheRegister is getting the drama. The thread doesn't seem that scathing to me. [0] It doesn't build, but the person who pointed it out also supplied a diff to make it happen. It also fails a few tests, but Paragon are more than happy to see if they can make it a bit more compliant. UBSan finds a few potential bugs, but again, Paragon are more than happy to fix the problems. There's some style guide suggestions, which Paragon seem to immediately take on board: > The patch will be splitted in v2 file-wise. Wasn't clear initially which way will be more convenient to review. [0] https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d74@paragon-software.com/ https://lore.kernel.org/lkml/2911ac5cd20b46e397be506268718d7...
- Delk 6y ago
  Yeah, The Register can often be witty and irreverent, and I wouldn't complain about that, but that kind of wit only works if there's an actual point behind it. Jumping on the apparently popular bandwagon of squeezing out "drama" from the Linux kernel mailing list isn't adding anything of value. It also seems a bit ignorant to call the submission "half-baked" just because it needed to be worked on and because someone pointed that out (also) in a slightly irreverent way. All of those points you bring up about the feedback they got seem like just business as usual for a larger merge of code to a carefully developed FOSS project.
- cortesoft 6y ago
  Yeah, this seems like one of those 'dramas' that incites bystanders more than the actual participants.
6y ago

Tade0 6y ago

I solemnly swear never to complain again about the size of PRs given to me for review. It's going to take months to throughly assess the quality of this.

Havoc 6y ago

Ouch. That looks really useful, but at the same time possibly their first kernel submission perhaps? Hope I’m wrong but I think they’ll be fighting initial bad impression for a while despite good intention

cactus2093 6y ago

It's strange to me that there's no file system with basic features like journaling and support for files larger than a couple of GB, that is supported across all major desktop OSes (MacOS, Windows, Linux, and FreeBSD). All platforms support the same i/o standards like USB or DisplayPort, why did filesystems never make the cut to become a cross-system standard? Imagine if you could have a backup drive (with reasonable modern data protections) that you could just plug into different systems and save all your files to. Isn't it odd that such a simple thing isn't possible? I guess network attached storage has gotten pretty accessible at this point so there's no need for it?

quarantine 6y ago
There's a ZFS driver for Linux, macOS, and Windows.
- wtallis 6y ago
  In all three cases you list, it's a third-party module providing support, rather than it being a standard feature of the OS that can be expected to be generally available on more than 1% of the install base.
- hoistbypetard 6y ago
  ZFS development moves so fast that it is common for my (FreeBSD-based) FreeNAS box to warn me when I upgrade my OS that certain actions will make it incompatible with the prior version of FreeNAS. That is fine and appropriate for a drive that will be connected to the system for the foreseeable future. That kind of compatibility concern makes me squeamish about using ZFS for a drive that I want to share between different systems. If it's easy to make it incompatible between two releases for the same system, that smells like a waiting nightmare trying to keep it compatible between Linux, FreeBSD and Windows.
  - DenseComet 6y ago
    Yeah it should stabilize in the next couple of months with the release of OpenZFS 2.0 as that release is supposed to signify the unification of ZFS on Linux and FreeBSD. ZFS on FreeBSD is being rebased onto ZFS on Linux. Theres also been some talk on adding MacOS zfs support OpenZFS but thats still up in the air.

beervirus 6y ago

27k is 27 kB. This is 27 kLOC.

prirun 6y ago

It's just a single data point, but I tried Paragon's ext4 driver (the paid version) for Mac many years ago. It seemed to work, but when the drive was connected back to Linux, there were all kinds of fsck errors. Immediately deleted it.

apfsx 6y ago
This has also been my exact experience. I used Paragon's NTFS driver (paid) for Mac for my external SSD. After using, when plugging into Windows it would always find "errors" and recommend to Scan and Fix them.
- Wowfunhappy 6y ago
  Fwiw, I had the same problem with Paragon but have had a lot more success with Tuxera’s driver, if you’re still looking for a solution.

adamretter 6y ago

I hope the it's better than their ext2 drivers for macOS. I have had nothing but problems and data loss and their support is the worst.

jug 6y ago
Same on Windows regarding data loss. Terrible and I’ll never use their ext2 drivers again.

deleted 6y ago

[deleted]

aaron695 6y ago

> "It looks as though that with NTFS being surpassed by other more advanced file-systems, Is this remotly true? Will my grandmother not be using NTFS in 10 years?

wtallis 6y ago
Microsoft has been trying to come up with a replacement for NTFS for a very long time. They've had mixed success with trying to extend NTFS with more advanced features like the now-deprecated TxF. There's little doubt that even its creators see NTFS as something of a dead-end. Whether your grandmother is still using it in 10 years depends primarily on whether Microsoft can get its act together to pick and ship a replacement.
- aaron695 6y ago
  > Whether your grandmother is still using it in 10 years depends primarily on whether Microsoft can get its act together to pick and ship a replacement. So yes she will I guess. Then it's vital Linux gets NTFS working well. My partner is not going to use Linux things if every time they try and transfer the 8k 3D holographic photos of our CRISPR'ed dog learning to spell to my grandmother it doesn't work on her Holovision. True story, last month I just lost about 1 in 10 of my media files on my Linux Share to NTFS issues. So now I run the box on Windows.
  - wtallis 6y ago
    > True story, last month I just lost about 1 in 10 of my media files on my Linux Share to NTFS issues. So now I run the box on Windows. Were you seriously running a Linux-based NAS with NTFS as the underlying filesystem, or did you mean something very different? I can't imagine why anyone would ever think their choice of disk filesystem on a server—hiding behind a network filesystem—should be influenced by what disk filesystems are supported by client devices.
    - aaron695 6y ago
      Given my original question about my grandmother has been downvoted it says it all about the judginess of Linux users ;) My setup was - Windows and Ubuntu dual boot gaming PC with a NTFS 4G external hard disk, Windows network share. Default boot was to Ubuntu. Torrents running off wireless laptop to the network share. Amazon Fire stick with Kodi wireless running to network share.

PhoenixRobo 6y ago

Does it need to be included in the Linux Kernel? I understand that NTFS is 27 years old and mostly to support Microsoft file systems. Can't the driver be an external download?

EE84M3i 6y ago
In-tree drivers are the standard for Linux kernel modules.

bArray 6y ago

27k lines isn't that crazy. I've merged larger patches, it just takes a while. This is a very negative view of an open source contribution.

wtallis 6y ago
> I've merged larger patches, In what kind of context? The Linux kernel? Filesystem code is pretty tricky to begin with, and prone to very subtle bugs with very not-subtle consequences. And this isn't greenfield development of a new filesystem, but an implementation that needs to remain highly compatible with Microsoft's version. This FS driver has to be maintained to track changes to two operating systems. So this 27kloc can reasonably be expected to encompass a lot more complexity than your average 27kloc, and it requires a lot more review effort than something like 27kloc of GPU driver register definitions.
- bArray 6y ago
  > In what kind of context? The Linux kernel? Not the Linux kernel, but a large embedded system. > Filesystem code is pretty tricky to begin with, and prone > to very subtle bugs with very not-subtle consequences. This code has been running in the wild for quite a while now, it has had a trial by fire. And there's no way around testing, subtle ext4 bugs still crop up despite the maturity of the filesystem. > And this isn't greenfield development of a new filesystem, > but an implementation that needs to remain highly > compatible with Microsoft's version. No amount of code review will stop Microsoft from adapting their version. Also, I doubt Microsoft themselves will change too much about the filesystem given the compatibility they themselves have to maintain with cold storage NTFS drives. > This FS driver has to be maintained to track changes to > two operating systems. You make it sound as if Microsoft have a hand in any of this. Also, have you seen the state of the current NTFS driver? It's a bit flakey (no disrespect to the maintainers).
  - wtallis 6y ago
    > No amount of code review will stop Microsoft from adapting their version. Way to miss the point. Code review for the kernel isn't just about verifying that the code currently works. It's also about making sure the code is maintainable. Microsoft is relevant here because their actions will increase the maintenance burden of any Linux NTFS driver. Kernel developers rightly need to be concerned about how difficult it will be to extend the NTFS driver to handle new NTFS features that Microsoft introduces.

userbinator 6y ago

I wonder if the reason for the huge line count is overly verbose code, or if it's just the inherent complexity of NTFS. For contrast, I wrote a FAT32 filesystem driver (read/write) for an embedded system a long time ago, and it was less than 1K lines --- of Asm.

The_Colonel 6y ago
NTFS is way more complicated (and feature rich) filesystem than (rather barebones) FAT32.

r41nbowdash 6y ago

200loc/h, it's 4-5 weeks of work for a single person