5 ms·

I don't mean for this question to sound inflammatory but I can't help but think: why does this matter? You can already use exFAT on Linux via FUSE, and as point

by hnarn 7y ago

I don't mean for this question to sound inflammatory but I can't help but think: why does this matter? You can already use exFAT on Linux via FUSE, and as pointed out by VentureBeat: > To be clear, Microsoft isn’t open-sourcing exFAT — but it is making sure anyone building with Linux can use it. 'It’s important to us that the Linux community can make use of exFAT included in the Linux kernel with confidence.' exFAT is still not open source, and still can't be distributed with the kernel. So what changed? It will now be easier to integrate it into a custom kernel instead of using FUSE? Why is this better than the current situation, where you'd install exfat-utils for example? Are there benefits of using a kernel module that I'm missing which makes this important news?

vengefulduck 7y ago
But they can put their own implementation of exfat inside the kernel now without worrying about patents, which is what previously prevented it from being included. I will also help that they will have an official spec to go off of now, so they won’t have to waste time with reverse engineering (plus the legal concerns that comes with that).
8K832d7tNmiQ 7y ago
legality and implementation straight on linux kernel would be the first thing I'd say.
gvb 7y ago
According to John Gossman "Microsoft Distinguished Engineer & Linux Foundation Board Member" in a related announcement, Microsoft plans to contribute it to the Linux kernel: We also support the eventual inclusion of a Linux kernel with exFAT support in a future revision of the Open Invention Network’s Linux System Definition, where, once accepted, the code will benefit from the defensive patent commitments of OIN’s 3040+ members and licensees. Still some patent squirrelly words (they imply you have to sign a patent cross licensing agreement with the OIN - not sure of the implications) and no timeline. Ref: https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat-linux-kernel/ https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat... OIN Ref: https://www.openinventionnetwork.com/joining-oin/ https://www.openinventionnetwork.com/joining-oin/
- WorldMaker 7y ago
  Most of the major Linux distributions are directly members or licensees of the OIN already. It looks like any distribution that isn't can get a license by filling out a form, no money needed and royalty-free. To my reading, the OIN is a "mutually assured destruction" organization for patents around Linux that owns some of the patents outright and basically tries to insure that if any member sues another member or licensee over any of their patents the OIN itself and all the other members should band together and counter-sue over the remainder of the patents. It seems an interesting nuclear option to protect Linux from patent lawsuits.
  - Boulth 7y ago
    The list definitely is long but I don't see Debian or Archlinux there: https://www.openinventionnetwork.com/community-of-licensees/ https://www.openinventionnetwork.com/community-of-licensees/
- ldng 7y ago
  Well at least they did not deny patent issues. As for the driver, a community one is already on the way so they'll probably just piggyback on it. Or, if they want to contribute an official driver, they should have contribute it at the same time as the spec, IMO.
garaetjjte 7y ago
FUSE is very slow compared to kernel module.
- fooker 7y ago
  Citation needed.
  - 693471 7y ago
    I don't think we need a citation for common knowledge about how OS kernels work
  - mackman 7y ago
    I’ve been building SD card based Pi devices and the limiting factor in IO perf is the FUSE exFat implementation. There’s a leaked Samsung internal implementation that is over 2x as fast in my benchmarks. I can’t attest whether it’s the fact it’s FUSE or just other performance optimizations that is the reason though.
    - rahuldottech 7y ago
      The leaked kernel was apparently later open sourced [1], although I can't actually find it on the official samsung website. [1]: https://www.phoronix.com/scan.php?page=news_item&px=MTQzODQ https://www.phoronix.com/scan.php?page=news_item&px=MTQzODQ
  - mises 7y ago
    Doing things in-kernel is almost always faster. No dealing with complicated context-switching, moving memory back-and-forth, etc. I can't necessarily provide a formal, data-backed citation, but it's pretty well understood.
cesarb 7y ago
> why does this matter? You can already use exFAT on Linux via FUSE It means that exFAT will be available everywhere. For instance, right now if you want exFAT support on Fedora or RHEL, you have to enable rpmfusion and install fuse-exfat and exfat-utils from there; once it's in the upstream kernel, it will be available on a default install.
gregkh 7y ago
It will be in the "real" kernel soon, here is a patch I submitted adding it: https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@linuxfoundation.org/ https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@li... And yes, this is _much_ better than using a FUSE interface to the filesystem.
- ghostpepper 7y ago
  For the 99% of people reading who don't know the kernel as well as you (seriously) can you explain why this is the case? Does a userspace filesystem driver lose a lot of performance to context switching, or is there something unique to filesystems that slows them down in user space?
  - gregkh 7y ago
    > Does a userspace filesystem loose a lot of performance... Yes. Try the two versions out, and see for yourself if you are curious.
    - dekhn 7y ago
      I use a FUSE filesystem on linux- not ExFAT, but as an S3 interface. I often see 1+Gigabyte/sec throughput over the wire from simple multithreaded IO operations. FUSE can be very fast.
      - notriddle 7y ago
        For one thing, that's throughput, not latency. For another, that's probably a desktop, not a smartphone.
        dekhn 7y ago
        That's correct, my metric for performance is throughput I don't really think a desktop vs a smartphone matters much at this comparison, it's just my observation that when people say a fuse filesystem is slow it's just not well-engineered, not that the fuse API is inherently slow
thetinguy 7y ago
exFAT will also be available on other devices like cameras android phones
- yc12340 7y ago
  Care to explain how? Microsoft isn't giving away it's exFAT patents. They are merely suggesting, that everyone licensed them at Microsoft's own terms. Those terms include becoming a member of Microsoft-sponsored patent ring and agreeing to it's terms and conditions: https://www.openinventionnetwork.com/joining-oin/oin-license-agreement/ https://www.openinventionnetwork.com/joining-oin/oin-license.... The conditions can be changed anytime (!!) and the patent license terminates if you sue Microsoft or any other OIN member for any patent violations whatsoever. Nikon, ZTE, Xiaomi, Samsung and thousands of others (basically all, who were or still are patent-trolled by Microsoft) haven't magically become members of OIN, and it's unclear if they ever will. Google and Red Hat are members, but neither produces any devices with external SD-cards...
zamalek 7y ago
The big deal here is that they are placing the patent into OIN - I'm probably correct in guessing that it is now the most powerful patent in the OIN arsenal. More electronics/software shops are going to be pressured to join the OIN. Consider sharing files on physical media and, oh, that UEFI basically uses FAT. Patent trolls are probably collectively experiencing a migraine after this news.
- paule89 7y ago
  Also hope that uefi will soon use exfat. Still better than plain old fat.
gambiting 7y ago
Because Fuse is slooow - I've had an external Exfat drive mounted on Raspberry Pi 4 and writing large amount of data would pretty much max out the CPU. Ext3/4 has no such issue, I can write to it for days without taxing the CPU. Apparently it's a known problem with Fuse partitions, NFTS-3G has the same problem.

by hnarn 7y ago

vengefulduck 7y ago

But they can put their own implementation of exfat inside the kernel now without worrying about patents, which is what previously prevented it from being included. I will also help that they will have an official spec to go off of now, so they won’t have to waste time with reverse engineering (plus the legal concerns that comes with that).

8K832d7tNmiQ 7y ago

legality and implementation straight on linux kernel would be the first thing I'd say.

gvb 7y ago

According to John Gossman "Microsoft Distinguished Engineer & Linux Foundation Board Member" in a related announcement, Microsoft plans to contribute it to the Linux kernel: We also support the eventual inclusion of a Linux kernel with exFAT support in a future revision of the Open Invention Network’s Linux System Definition, where, once accepted, the code will benefit from the defensive patent commitments of OIN’s 3040+ members and licensees. Still some patent squirrelly words (they imply you have to sign a patent cross licensing agreement with the OIN - not sure of the implications) and no timeline. Ref: https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat-linux-kernel/ https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat... OIN Ref: https://www.openinventionnetwork.com/joining-oin/ https://www.openinventionnetwork.com/joining-oin/

WorldMaker 7y ago
Most of the major Linux distributions are directly members or licensees of the OIN already. It looks like any distribution that isn't can get a license by filling out a form, no money needed and royalty-free. To my reading, the OIN is a "mutually assured destruction" organization for patents around Linux that owns some of the patents outright and basically tries to insure that if any member sues another member or licensee over any of their patents the OIN itself and all the other members should band together and counter-sue over the remainder of the patents. It seems an interesting nuclear option to protect Linux from patent lawsuits.
- Boulth 7y ago
  The list definitely is long but I don't see Debian or Archlinux there: https://www.openinventionnetwork.com/community-of-licensees/ https://www.openinventionnetwork.com/community-of-licensees/
ldng 7y ago
Well at least they did not deny patent issues. As for the driver, a community one is already on the way so they'll probably just piggyback on it. Or, if they want to contribute an official driver, they should have contribute it at the same time as the spec, IMO.

garaetjjte 7y ago

FUSE is very slow compared to kernel module.

fooker 7y ago
Citation needed.
- 693471 7y ago
  I don't think we need a citation for common knowledge about how OS kernels work
- mackman 7y ago
  I’ve been building SD card based Pi devices and the limiting factor in IO perf is the FUSE exFat implementation. There’s a leaked Samsung internal implementation that is over 2x as fast in my benchmarks. I can’t attest whether it’s the fact it’s FUSE or just other performance optimizations that is the reason though.
  - rahuldottech 7y ago
    The leaked kernel was apparently later open sourced [1], although I can't actually find it on the official samsung website. [1]: https://www.phoronix.com/scan.php?page=news_item&px=MTQzODQ https://www.phoronix.com/scan.php?page=news_item&px=MTQzODQ
- mises 7y ago
  Doing things in-kernel is almost always faster. No dealing with complicated context-switching, moving memory back-and-forth, etc. I can't necessarily provide a formal, data-backed citation, but it's pretty well understood.

cesarb 7y ago

> why does this matter? You can already use exFAT on Linux via FUSE It means that exFAT will be available everywhere. For instance, right now if you want exFAT support on Fedora or RHEL, you have to enable rpmfusion and install fuse-exfat and exfat-utils from there; once it's in the upstream kernel, it will be available on a default install.

gregkh 7y ago

It will be in the "real" kernel soon, here is a patch I submitted adding it: https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@linuxfoundation.org/ https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@li... And yes, this is _much_ better than using a FUSE interface to the filesystem.

ghostpepper 7y ago
For the 99% of people reading who don't know the kernel as well as you (seriously) can you explain why this is the case? Does a userspace filesystem driver lose a lot of performance to context switching, or is there something unique to filesystems that slows them down in user space?
- gregkh 7y ago
  > Does a userspace filesystem loose a lot of performance... Yes. Try the two versions out, and see for yourself if you are curious.
  - dekhn 7y ago
    I use a FUSE filesystem on linux- not ExFAT, but as an S3 interface. I often see 1+Gigabyte/sec throughput over the wire from simple multithreaded IO operations. FUSE can be very fast.
    - notriddle 7y ago
      For one thing, that's throughput, not latency. For another, that's probably a desktop, not a smartphone.
      - dekhn 7y ago
        That's correct, my metric for performance is throughput I don't really think a desktop vs a smartphone matters much at this comparison, it's just my observation that when people say a fuse filesystem is slow it's just not well-engineered, not that the fuse API is inherently slow

thetinguy 7y ago

exFAT will also be available on other devices like cameras android phones

yc12340 7y ago
Care to explain how? Microsoft isn't giving away it's exFAT patents. They are merely suggesting, that everyone licensed them at Microsoft's own terms. Those terms include becoming a member of Microsoft-sponsored patent ring and agreeing to it's terms and conditions: https://www.openinventionnetwork.com/joining-oin/oin-license-agreement/ https://www.openinventionnetwork.com/joining-oin/oin-license.... The conditions can be changed anytime (!!) and the patent license terminates if you sue Microsoft or any other OIN member for any patent violations whatsoever. Nikon, ZTE, Xiaomi, Samsung and thousands of others (basically all, who were or still are patent-trolled by Microsoft) haven't magically become members of OIN, and it's unclear if they ever will. Google and Red Hat are members, but neither produces any devices with external SD-cards...

zamalek 7y ago

The big deal here is that they are placing the patent into OIN - I'm probably correct in guessing that it is now the most powerful patent in the OIN arsenal. More electronics/software shops are going to be pressured to join the OIN. Consider sharing files on physical media and, oh, that UEFI basically uses FAT. Patent trolls are probably collectively experiencing a migraine after this news.

paule89 7y ago
Also hope that uefi will soon use exfat. Still better than plain old fat.

gambiting 7y ago

Because Fuse is slooow - I've had an external Exfat drive mounted on Raspberry Pi 4 and writing large amount of data would pretty much max out the CPU. Ext3/4 has no such issue, I can write to it for days without taxing the CPU. Apparently it's a known problem with Fuse partitions, NFTS-3G has the same problem.