4 ms·

> Brauner said that he thinks bcachefs is in "excellent shape to be upstreamed", but he is concerned with the number of filesystems in the kernel; he is glad to

> Brauner said that he thinks bcachefs is in "excellent shape to be upstreamed", but he is concerned with the number of filesystems in the kernel; he is glad to see that there are efforts to remove some of them. Changes that impact all of the filesystems in the tree "get painful very very fast" and, in some cases, there is no one available to review the changes. He would like the acceptance process to be more conservative; accepting NTFS/NTFS3 was "a huge mistake", for example. As someone not familiar with the filesystem related parts of the kernel, it's quite surprising to hear this. It sounds like filesystems are a lot more integrated (or at least less modular) than other kernel subsystems. Anyone else who found this surprising, I recommend reading this other LWN article that is referenced as well: https://lwn.net/Articles/886708/ https://lwn.net/Articles/886708/ However, that mostly discusses issues caused by having 32-bit data structures, and how they will cause issues in 2038 when it's no longer able to handle the timestamps required. Specifically for ext3, NTFS, and ReiserFS. But other than that issue, I don't really understand why it's difficult to simply rip out support for a specific filesystem. Compared to something like a driver for a PCIe or USB device, what makes filesystems so much more integrated and difficult to remove?

djbusby 3y ago
Hard to maintain because all this code has to handle common thing (file, directory, etc) in unique way. A big mapping mess. Hard to remove because someone, somewhere uses it and Linux doesn't like to break things for the user.
wmf 3y ago
I don't know that it's technically difficult, but Linux hates breaking backward compatibility. If they remove a filesystem, some users won't be able to mount their filesystems any more. But Linux doesn't want to have unmaintained code either, so they only accept a filesystem if it's going to be maintained for the next 10-20 years.
- Tuna-Fish 3y ago
  The only "easy path" is if there exists a working fuse implementation of the same filesystem. Then there is a good fallback path for the users who still need support.
  - Dylan16807 3y ago
    You can always run the kernel implementation as a fuse filesystem with guestmount. Side note: I feel almost gaslit, it's really hard to find any mention of guestmount running a VM outside this single page https://libguestfs.org/guestfs-internals.1.html https://libguestfs.org/guestfs-internals.1.html
    - heavyset_go 3y ago
      guestfs has some really useful tools that are, as you've noted, seemingly esoteric.
    - pengaru 3y ago
      That's an interesting approach. It's too bad linux can't already run any in-kernel filesystem as a user process via FUSE, when you prefer greater isolation in exchange for worse performance, at the flip of a mount option. There's no technical reason for this to not be possible IMO... it's just a product of the tight coupling of everything in-kernel, as implemented today. I'm short on time to confirm at the moment, but I believe it was this [0] talk that left me with the impression kernel devs were exploring general solutions of this nature. [0] https://www.youtube.com/watch?v=xjv8Jv58bMs https://www.youtube.com/watch?v=xjv8Jv58bMs
    - yencabulator 3y ago
      Except in this case there will no longer be such a thing as "the kernel implementation". You'd have to run an older kernel, too. That's a ticking time bomb, more a migration/recovery strategy than something one could recommend longer term.
      - AshamedCaptain 3y ago
        It's really not a "ticking time bomb". The older kernel is only running as a guest in a VM. It will eventually stop working (e.g. when Intel changes x86), but it's not really a security issue.
cesarb 3y ago
> As someone not familiar with the filesystem related parts of the kernel, it's quite surprising to hear this. It sounds like filesystems are a lot more integrated (or at least less modular) than other kernel subsystems. AFAIK, on Linux filesystems are closely coupled with the memory management subsystem and the directory and inode caches. It's part of the reason why filesystem access on Linux is so fast. > But other than that issue, I don't really understand why it's difficult to simply rip out support for a specific filesystem. Compared to something like a driver for a PCIe or USB device, what makes filesystems so much more integrated and difficult to remove? It's not that unusual for users to have a partition containing a filesystem (sometimes, but not always, on an external drive) surviving unchanged through several migrations to newer Linux distributions and/or hardware. Compared to something like a PCIe or USB device, filesystems have a longer life.
- tyingq 3y ago
  Buffer cache also. An article that goes into detail with an example of the coupling you're describing: https://lwn.net/Articles/930173/ https://lwn.net/Articles/930173/
  - klooney 3y ago
    Ext2/ext4 is a little special that way, most aren't quite so intertwined.
    - tyingq 3y ago
      The article mentions the proposed change would also affect these: "including the ext4 filesystem, but also F2FS, FAT, GFS2, HFS, ISO9660 (CDROM), JFS, NTFS, NTFS3, and the device-mapper layer"
- yencabulator 3y ago
  That's more of a reason why it's hard to change the core MM/page cache/etc logic, as you have to change all the users of it (every filesystem, etc). Ripping out a filesystem is, technically, near-trivial. The downsides are fully human: the Linux kernel-userspace ABI is normally considered a golden promise that shall not be broken. The developers don't want users to have a bad morning on which their old filesystems no longer work.
mike_hock 3y ago
It surprised me too. I'd have expected the infra code to be lacking when there's only 2 or 3 filesystems in the kernel, but after multiple filesystems had been added, I'd have expected the kinks to be ironed out and the common code to be generic enough to support almost any filesystem.

djbusby 3y ago

Hard to maintain because all this code has to handle common thing (file, directory, etc) in unique way. A big mapping mess. Hard to remove because someone, somewhere uses it and Linux doesn't like to break things for the user.

wmf 3y ago

I don't know that it's technically difficult, but Linux hates breaking backward compatibility. If they remove a filesystem, some users won't be able to mount their filesystems any more. But Linux doesn't want to have unmaintained code either, so they only accept a filesystem if it's going to be maintained for the next 10-20 years.

Tuna-Fish 3y ago
The only "easy path" is if there exists a working fuse implementation of the same filesystem. Then there is a good fallback path for the users who still need support.
- Dylan16807 3y ago
  You can always run the kernel implementation as a fuse filesystem with guestmount. Side note: I feel almost gaslit, it's really hard to find any mention of guestmount running a VM outside this single page https://libguestfs.org/guestfs-internals.1.html https://libguestfs.org/guestfs-internals.1.html
  - heavyset_go 3y ago
    guestfs has some really useful tools that are, as you've noted, seemingly esoteric.
  - pengaru 3y ago
    That's an interesting approach. It's too bad linux can't already run any in-kernel filesystem as a user process via FUSE, when you prefer greater isolation in exchange for worse performance, at the flip of a mount option. There's no technical reason for this to not be possible IMO... it's just a product of the tight coupling of everything in-kernel, as implemented today. I'm short on time to confirm at the moment, but I believe it was this [0] talk that left me with the impression kernel devs were exploring general solutions of this nature. [0] https://www.youtube.com/watch?v=xjv8Jv58bMs https://www.youtube.com/watch?v=xjv8Jv58bMs
  - yencabulator 3y ago
    Except in this case there will no longer be such a thing as "the kernel implementation". You'd have to run an older kernel, too. That's a ticking time bomb, more a migration/recovery strategy than something one could recommend longer term.
    - AshamedCaptain 3y ago
      It's really not a "ticking time bomb". The older kernel is only running as a guest in a VM. It will eventually stop working (e.g. when Intel changes x86), but it's not really a security issue.

cesarb 3y ago

> As someone not familiar with the filesystem related parts of the kernel, it's quite surprising to hear this. It sounds like filesystems are a lot more integrated (or at least less modular) than other kernel subsystems. AFAIK, on Linux filesystems are closely coupled with the memory management subsystem and the directory and inode caches. It's part of the reason why filesystem access on Linux is so fast. > But other than that issue, I don't really understand why it's difficult to simply rip out support for a specific filesystem. Compared to something like a driver for a PCIe or USB device, what makes filesystems so much more integrated and difficult to remove? It's not that unusual for users to have a partition containing a filesystem (sometimes, but not always, on an external drive) surviving unchanged through several migrations to newer Linux distributions and/or hardware. Compared to something like a PCIe or USB device, filesystems have a longer life.

tyingq 3y ago
Buffer cache also. An article that goes into detail with an example of the coupling you're describing: https://lwn.net/Articles/930173/ https://lwn.net/Articles/930173/
- klooney 3y ago
  Ext2/ext4 is a little special that way, most aren't quite so intertwined.
  - tyingq 3y ago
    The article mentions the proposed change would also affect these: "including the ext4 filesystem, but also F2FS, FAT, GFS2, HFS, ISO9660 (CDROM), JFS, NTFS, NTFS3, and the device-mapper layer"
yencabulator 3y ago
That's more of a reason why it's hard to change the core MM/page cache/etc logic, as you have to change all the users of it (every filesystem, etc). Ripping out a filesystem is, technically, near-trivial. The downsides are fully human: the Linux kernel-userspace ABI is normally considered a golden promise that shall not be broken. The developers don't want users to have a bad morning on which their old filesystems no longer work.

mike_hock 3y ago

It surprised me too. I'd have expected the infra code to be lacking when there's only 2 or 3 filesystems in the kernel, but after multiple filesystems had been added, I'd have expected the kinks to be ironed out and the common code to be generic enough to support almost any filesystem.