5 ms·

The older I get the more I feel like anything other than the ExtantFS family is just silly. The filesystem should do files, if you want something more complex

The older I get the more I feel like anything other than the ExtantFS family is just silly. The filesystem should do files, if you want something more complex do it in userspace. We even have FUSE if you want to use the Filesystem API with your crazy network database thing.

_zagj 1y ago
> The older I get the more I feel like anything other than the ExtantFS family is just silly. The extended (not extant) family (including ext4) don't support copy-on-write. Using them as your primary FS after 2020 (or even 2010) is like using a non-journaling file system after 2010 (or even 2001)--it's a non-negotiable feature at this point. Btrfs has been stable for a decade, and if you don't like or trust it, there's always ZFS, which has been stable 20 years now. Apple now has AppFS, with CoW, on all their devices, while MSFT still treats ReFS as unstable, and Windows servers still rely heavily on NTFS.
- msgodel 1y ago
  Again I don't really want the kernel managing a database for me like that, the few applications that need that can do it themselves just fine. (IME mostly just RDBMSs and Qemu.)
- robotnikman 1y ago
  >Windows will at some point have ReFS They seem to be slowly introducing it to the masses, Dev drives you set up on Windows automatically use ReFS
- milkey_mouse 1y ago
  Hell, there's XFS if you love stability but want CoW.
  - josephcsible 1y ago
    XFS doesn't support whole-volume snapshots, which is the main reason I want CoW filesystems. And it also stands out as being basically the only filesystem that you can't arbitrarily shrink without needing to wipe and reformat.
    - leogao 1y ago
      you can always have an LVM layer for atomic snapshots
      - josephcsible 1y ago
        There are advantages to having the filesystem do the snapshots itself. For example, if you have a really big file that you keep deleting and restoring from a snapshot, you'll only pay the cost of the space once with Btrfs, but will pay it every time over with LVM.
      - shtripok 1y ago
        On some of my zfs servers, the number of snapshots (mostly periodic, rotated — hour, day, month, updates, data maintenance work) is 10-12 thousand. LVM can't do that.
    - kzrdude 1y ago
      there was the "old dog new tricks" xfs talk long time ago, but I suppose it was for fun and exploration and not really a sneak peek into snapshots
    - MertsA 1y ago
      You can shrink XFS, but only the realtime volume. All you need is xfs_db and a steady hand. I once had to pull this off for a shortened test program for a new server platform at Meta. Works great except some of those filesystems did somehow get this weird corruption around used space tracking that xfs_repair couldn't detect... It was mostly fine.
    - adrian_b 1y ago
      Many years ago, XFS did not support snapshots. However, there is also a long time since XFS supports snapshots. See for example: https://thelinuxcode.com/xfs-snapshot/ https://thelinuxcode.com/xfs-snapshot/ I am not sure what you mean by "whole-volume" snapshots, but I have not noticed any restrictions in the use of the XFS snapshots. As expected, they store a snapshot of the entire file system, which can be restored later. In many decades of managing computers with all kinds of operating systems and file systems, on a variety of servers and personal computers, I have never had the need to shrink a file system. I cannot imagine how such a need can arise, except perhaps as a consequence of bad planning. There are also many decades since I have deprecated the use of multiple partitions on a storage device, with the exception of bootable devices, which must have a dedicated partition for booting, conforming to the BIOS or UEFI expectations. For anything that was done in the ancient times with multiple partitions there are better alternatives now. With the exception of bootable USB sticks with live Linux or FreeBSD partitions, I use XFS on whole SSDs or HDDs (i.e. unpartitioned), regardless if they are internal or external, so there is never any need for changing the size of the file system. Even so, copying a file system to an external device, reformatting the device and copying the file system back is not likely to be significantly slower than shrinking in place. In fact sometimes it can be faster and it has the additional benefit that the new copy of the file system will be defragmented. Much more significant than the lack of shrinking ability, which may slow down a little something that occurs very seldom, is that both EXT4 and XFS are much faster for most applications than the other file systems available for Linux, so they are fast for the frequent operations. You may choose another file system for other reasons, but choosing it for making faster a very rare operation like shrinking is a very weak reason.
- leogao 1y ago
  btrfs has eaten my data within the last decade. (not even because of the broken erasure coding, which I was careful to avoid!) not sure I'm willing to give it another chance. I'd much rather use zfs.
  - bombcar 1y ago
    I used reiserfs for awhile after I noticed it eating data (tail packing for the power loss) but quickly switched to xfs when it became available. Speed is sometimes more important than absolute reliability, but it’s still an undesirable tradeoff.
- NewJazz 1y ago
  CoW is an efficiency gain. Does it do anything to ensure data integrity, like journaling does? I think it is an unreasonable comparison you are making.
  - webstrand 1y ago
    I use CoW a lot just managing files. It's only an efficiency gain if you have enough space to do the data-copying operation. And that's not necessarily true in all cases. Being able to quickly take a "backup" copy of some multi-gb directory tree before performing some potentially destructive operation on it is such a nice safety net to have. It's also a handy way to backup file metadata, like mtime, without having to design a file format for mapping saved mtimes back to their host files.
  - _zagj 1y ago
    > CoW is an efficiency gain. You're thinking of the optimization technique of CoW, as in what Linux does when spawning a new thread or forking a process. I'm talking about it in the context of only ever modifying copies of file system data and metadata blocks, for the purpose of ensuring file system integrity, even in the context of sudden power loss (EDIT: wrong link): https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.sys_arch/topic/fsys_COW_filesystem.html https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino... If anything, ordinary file IO is likely to be slightly slower on a CoW file system, due to it always having to copy a block before said block can be modified and updating block pointers.
  - throw0101d 1y ago
    > Does it do anything to ensure data integrity, like journaling does? What kind of journaling though? By default ext4 only uses journaling for metadata updates, not data updates (see "ordered" mode in ext4(5)). So if you have a (e.g.) 1000MB file, and you update 200MB in the middle of it, you can have a situation where the first 100MB is written out and the system dies with the other 100MB vanishing. With a CoW, if the second 100MB is not written out and the file sync'd, then on system recovery you're back to the original file being completely intact. With ext4 in the default configuration you have a file that has both new-100MB and stale-100MB in the middle of it. The updating of the file data and the metadata are two separate steps (by default) in ext4: * https://www.baeldung.com/linux/ext-journal-modes https://www.baeldung.com/linux/ext-journal-modes * https://michael.kjorling.se/blog/2024/ext4-defaulting-to-data-ordered/ https://michael.kjorling.se/blog/2024/ext4-defaulting-to-dat... * https://fy.blackhats.net.au/blog/2024-08-13-linux-filesystems/ https://fy.blackhats.net.au/blog/2024-08-13-linux-filesystem... Whereas with a proper CoW (like ZFS), updates are ACID.
- tbrownaw 1y ago
  > The extended (not extant) family (including ext4) I read that more as "we have filesystems at home, and also get off my lawn".
- zahlman 1y ago
  ... NTFS does copy-on-write? ... It does hard links? After checking: It does hard links. ... Why didn't any programs I had noticeably take advantage of that?
  - _zagj 1y ago
    > NTFS does copy-on-write? No, it doesn't. Maybe you're thinking of shadow volume copies or something else. CoW files systems never modify data or metadata blocks directly, only modifying copies, with the root of the updated block pointer graph only updated after all other changes have been synced. Read this: https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.sys_arch/topic/fsys_COW_filesystem.html https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino...
    - zahlman 1y ago
      >No, it doesn't. Maybe you're thinking of shadow volume copies or something else. I was asking, because didn't know, and I thought the other person was implying that it did. I know what copy-on-write is.
      - _zagj 1y ago
        The "other person" (only mention of NTFS) is me, here: > while MSFT still treats ReFS as unstable, and Windows servers still rely heavily on NTFS. By this I implied it's an embarrassment to MSFT that iOS devices have a better, more reliable file system (AppFS) than even Windows servers now (having to rely on NTFS until ReFS is ready for prime time). If HN users and mods didn't tone-police so heavily, I could state things more frankly.
yjftsjthsd-h 1y ago
I mean, I'd really like some sort of data error detection (and ideally correction). If a disk bitflips one of my files, ext* won't do anything about it.
- timewizard 1y ago
  > some sort of data error detection (and ideally correction). That's pretty much built into most mass storage devices already. > If a disk bitflips one of my files The likelihood and consequence of this occurring is in many situations not worth the overhead of adding additional ECC on top of what the drive does. > ext* won't do anything about it. What should it do? Blindly hand you the data without any indication that there's a problem with the underlying block? Without an fsck what mechanism do you suppose would manage these errors as they're discovered?
  - throw0101d 1y ago
    >> > some sort of data error detection (and ideally correction). > That's pretty much built into most mass storage devices already. And ZFS has shown that it is not sufficient (at least for some use-cases, perhaps less of a big deal for 'residential' users). > The likelihood and consequence of this occurring is in many situations not worth the overhead of adding additional ECC on top of what the drive does. Not worth it to whom? Not having the option available at all is the problem. I can do a zfs set checksum=off pool_name/dataset_name if I really want that extra couple percentage points of performance. > Without an fsck what mechanism do you suppose would manage these errors as they're discovered? Depends on the data involved: if it's part of the file system tree metadata there are often multiple copies even for a single disk on ZFS. So instead of the kernel consuming corrupted data and potentially panicing (or going off into the weeds) it can find a correct copy elsewhere. If you're in a fancier configuration with some level of RAID, then there could be other copies of the data, or it could be rebuilt through ECC. With ext*, LVM, and mdadm no such possibility exists because there are no checksums at any of those layers (perhaps if you glom on dm-integrity?). And with ZFS one can set copies=2 on a per-dataset basis (perhaps just for /home?), and get multiple copies strewn across the disk: won't save you from a drive dying, but could save you from corruption.
heavyset_go 1y ago
Transparent compression, checksumming, copy-on-write, snapshots and virtual subvolumes should be considered the minimum default feature set for new OS installations in TYOOL 2025. You get that with APFS by default on macOS these days and those features come for free in btrfs, some in XFS, etc on Linux.
- riobard 1y ago
  APFS checksums only fs metadata not user data which is a pita. Presumably because APFS is used on single drive systems and there’s no redundancy to recover from anyway. Still, not ideal.
  - vbezhenar 1y ago
    Apple trusts their hardware to do their own checksums properly. Modern SSD uses checksums and parity codes for blocks. SATA/NVMe include checksums for protocol frames. The only unreliable component is RAM, but FS checksums can't help here, because RAM bit likely will be flipped before checksum is calculated or after checksum is verified.
    - riobard 1y ago
      If they do trust their hardware, APFS won’t need to checksum fs metadata either, so I guess they don’t trust it well enough? Also I have external drives that is not Apple sanctioned to store files and I don’t trust them enough either, and there’s no choice of user data checksumming at all.
      - 1over137 1y ago
        Apple does not care about your external non-Apple drives. In the slightest.
    - deleted 1y ago
      [deleted]
    - londons_explore 1y ago

_zagj 1y ago

> The older I get the more I feel like anything other than the ExtantFS family is just silly. The extended (not extant) family (including ext4) don't support copy-on-write. Using them as your primary FS after 2020 (or even 2010) is like using a non-journaling file system after 2010 (or even 2001)--it's a non-negotiable feature at this point. Btrfs has been stable for a decade, and if you don't like or trust it, there's always ZFS, which has been stable 20 years now. Apple now has AppFS, with CoW, on all their devices, while MSFT still treats ReFS as unstable, and Windows servers still rely heavily on NTFS.

msgodel 1y ago
Again I don't really want the kernel managing a database for me like that, the few applications that need that can do it themselves just fine. (IME mostly just RDBMSs and Qemu.)
robotnikman 1y ago
>Windows will at some point have ReFS They seem to be slowly introducing it to the masses, Dev drives you set up on Windows automatically use ReFS
milkey_mouse 1y ago
Hell, there's XFS if you love stability but want CoW.
- josephcsible 1y ago
  XFS doesn't support whole-volume snapshots, which is the main reason I want CoW filesystems. And it also stands out as being basically the only filesystem that you can't arbitrarily shrink without needing to wipe and reformat.
  - leogao 1y ago
    you can always have an LVM layer for atomic snapshots
    - josephcsible 1y ago
      There are advantages to having the filesystem do the snapshots itself. For example, if you have a really big file that you keep deleting and restoring from a snapshot, you'll only pay the cost of the space once with Btrfs, but will pay it every time over with LVM.
    - shtripok 1y ago
      On some of my zfs servers, the number of snapshots (mostly periodic, rotated — hour, day, month, updates, data maintenance work) is 10-12 thousand. LVM can't do that.
  - kzrdude 1y ago
    there was the "old dog new tricks" xfs talk long time ago, but I suppose it was for fun and exploration and not really a sneak peek into snapshots
  - MertsA 1y ago
    You can shrink XFS, but only the realtime volume. All you need is xfs_db and a steady hand. I once had to pull this off for a shortened test program for a new server platform at Meta. Works great except some of those filesystems did somehow get this weird corruption around used space tracking that xfs_repair couldn't detect... It was mostly fine.
  - adrian_b 1y ago
    Many years ago, XFS did not support snapshots. However, there is also a long time since XFS supports snapshots. See for example: https://thelinuxcode.com/xfs-snapshot/ https://thelinuxcode.com/xfs-snapshot/ I am not sure what you mean by "whole-volume" snapshots, but I have not noticed any restrictions in the use of the XFS snapshots. As expected, they store a snapshot of the entire file system, which can be restored later. In many decades of managing computers with all kinds of operating systems and file systems, on a variety of servers and personal computers, I have never had the need to shrink a file system. I cannot imagine how such a need can arise, except perhaps as a consequence of bad planning. There are also many decades since I have deprecated the use of multiple partitions on a storage device, with the exception of bootable devices, which must have a dedicated partition for booting, conforming to the BIOS or UEFI expectations. For anything that was done in the ancient times with multiple partitions there are better alternatives now. With the exception of bootable USB sticks with live Linux or FreeBSD partitions, I use XFS on whole SSDs or HDDs (i.e. unpartitioned), regardless if they are internal or external, so there is never any need for changing the size of the file system. Even so, copying a file system to an external device, reformatting the device and copying the file system back is not likely to be significantly slower than shrinking in place. In fact sometimes it can be faster and it has the additional benefit that the new copy of the file system will be defragmented. Much more significant than the lack of shrinking ability, which may slow down a little something that occurs very seldom, is that both EXT4 and XFS are much faster for most applications than the other file systems available for Linux, so they are fast for the frequent operations. You may choose another file system for other reasons, but choosing it for making faster a very rare operation like shrinking is a very weak reason.
leogao 1y ago
btrfs has eaten my data within the last decade. (not even because of the broken erasure coding, which I was careful to avoid!) not sure I'm willing to give it another chance. I'd much rather use zfs.
- bombcar 1y ago
  I used reiserfs for awhile after I noticed it eating data (tail packing for the power loss) but quickly switched to xfs when it became available. Speed is sometimes more important than absolute reliability, but it’s still an undesirable tradeoff.
NewJazz 1y ago
CoW is an efficiency gain. Does it do anything to ensure data integrity, like journaling does? I think it is an unreasonable comparison you are making.
- webstrand 1y ago
  I use CoW a lot just managing files. It's only an efficiency gain if you have enough space to do the data-copying operation. And that's not necessarily true in all cases. Being able to quickly take a "backup" copy of some multi-gb directory tree before performing some potentially destructive operation on it is such a nice safety net to have. It's also a handy way to backup file metadata, like mtime, without having to design a file format for mapping saved mtimes back to their host files.
- _zagj 1y ago
  > CoW is an efficiency gain. You're thinking of the optimization technique of CoW, as in what Linux does when spawning a new thread or forking a process. I'm talking about it in the context of only ever modifying copies of file system data and metadata blocks, for the purpose of ensuring file system integrity, even in the context of sudden power loss (EDIT: wrong link): https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.sys_arch/topic/fsys_COW_filesystem.html https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino... If anything, ordinary file IO is likely to be slightly slower on a CoW file system, due to it always having to copy a block before said block can be modified and updating block pointers.
- throw0101d 1y ago
  > Does it do anything to ensure data integrity, like journaling does? What kind of journaling though? By default ext4 only uses journaling for metadata updates, not data updates (see "ordered" mode in ext4(5)). So if you have a (e.g.) 1000MB file, and you update 200MB in the middle of it, you can have a situation where the first 100MB is written out and the system dies with the other 100MB vanishing. With a CoW, if the second 100MB is not written out and the file sync'd, then on system recovery you're back to the original file being completely intact. With ext4 in the default configuration you have a file that has both new-100MB and stale-100MB in the middle of it. The updating of the file data and the metadata are two separate steps (by default) in ext4: * https://www.baeldung.com/linux/ext-journal-modes https://www.baeldung.com/linux/ext-journal-modes * https://michael.kjorling.se/blog/2024/ext4-defaulting-to-data-ordered/ https://michael.kjorling.se/blog/2024/ext4-defaulting-to-dat... * https://fy.blackhats.net.au/blog/2024-08-13-linux-filesystems/ https://fy.blackhats.net.au/blog/2024-08-13-linux-filesystem... Whereas with a proper CoW (like ZFS), updates are ACID.
tbrownaw 1y ago
> The extended (not extant) family (including ext4) I read that more as "we have filesystems at home, and also get off my lawn".
zahlman 1y ago
... NTFS does copy-on-write? ... It does hard links? After checking: It does hard links. ... Why didn't any programs I had noticeably take advantage of that?
- _zagj 1y ago
  > NTFS does copy-on-write? No, it doesn't. Maybe you're thinking of shadow volume copies or something else. CoW files systems never modify data or metadata blocks directly, only modifying copies, with the root of the updated block pointer graph only updated after all other changes have been synced. Read this: https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.sys_arch/topic/fsys_COW_filesystem.html https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino...
  - zahlman 1y ago
    >No, it doesn't. Maybe you're thinking of shadow volume copies or something else. I was asking, because didn't know, and I thought the other person was implying that it did. I know what copy-on-write is.
    - _zagj 1y ago
      The "other person" (only mention of NTFS) is me, here: > while MSFT still treats ReFS as unstable, and Windows servers still rely heavily on NTFS. By this I implied it's an embarrassment to MSFT that iOS devices have a better, more reliable file system (AppFS) than even Windows servers now (having to rely on NTFS until ReFS is ready for prime time). If HN users and mods didn't tone-police so heavily, I could state things more frankly.

yjftsjthsd-h 1y ago

I mean, I'd really like some sort of data error detection (and ideally correction). If a disk bitflips one of my files, ext* won't do anything about it.

timewizard 1y ago
> some sort of data error detection (and ideally correction). That's pretty much built into most mass storage devices already. > If a disk bitflips one of my files The likelihood and consequence of this occurring is in many situations not worth the overhead of adding additional ECC on top of what the drive does. > ext* won't do anything about it. What should it do? Blindly hand you the data without any indication that there's a problem with the underlying block? Without an fsck what mechanism do you suppose would manage these errors as they're discovered?
- throw0101d 1y ago
  >> > some sort of data error detection (and ideally correction). > That's pretty much built into most mass storage devices already. And ZFS has shown that it is not sufficient (at least for some use-cases, perhaps less of a big deal for 'residential' users). > The likelihood and consequence of this occurring is in many situations not worth the overhead of adding additional ECC on top of what the drive does. Not worth it to whom? Not having the option available at all is the problem. I can do a zfs set checksum=off pool_name/dataset_name if I really want that extra couple percentage points of performance. > Without an fsck what mechanism do you suppose would manage these errors as they're discovered? Depends on the data involved: if it's part of the file system tree metadata there are often multiple copies even for a single disk on ZFS. So instead of the kernel consuming corrupted data and potentially panicing (or going off into the weeds) it can find a correct copy elsewhere. If you're in a fancier configuration with some level of RAID, then there could be other copies of the data, or it could be rebuilt through ECC. With ext*, LVM, and mdadm no such possibility exists because there are no checksums at any of those layers (perhaps if you glom on dm-integrity?). And with ZFS one can set copies=2 on a per-dataset basis (perhaps just for /home?), and get multiple copies strewn across the disk: won't save you from a drive dying, but could save you from corruption.

heavyset_go 1y ago

Transparent compression, checksumming, copy-on-write, snapshots and virtual subvolumes should be considered the minimum default feature set for new OS installations in TYOOL 2025. You get that with APFS by default on macOS these days and those features come for free in btrfs, some in XFS, etc on Linux.

riobard 1y ago
APFS checksums only fs metadata not user data which is a pita. Presumably because APFS is used on single drive systems and there’s no redundancy to recover from anyway. Still, not ideal.
- vbezhenar 1y ago
  Apple trusts their hardware to do their own checksums properly. Modern SSD uses checksums and parity codes for blocks. SATA/NVMe include checksums for protocol frames. The only unreliable component is RAM, but FS checksums can't help here, because RAM bit likely will be flipped before checksum is calculated or after checksum is verified.
  - riobard 1y ago
    If they do trust their hardware, APFS won’t need to checksum fs metadata either, so I guess they don’t trust it well enough? Also I have external drives that is not Apple sanctioned to store files and I don’t trust them enough either, and there’s no choice of user data checksumming at all.
    - 1over137 1y ago
      Apple does not care about your external non-Apple drives. In the slightest.
  - deleted 1y ago
    [deleted]
  - londons_explore 1y ago