3 ms·

Not everyone needs copy on write and checksumming, it doesn't make sense to waste CPU cycles on checksums when the use case doesn't require it. ext4 is a perfe

by tscs37 9y ago

Not everyone needs copy on write and checksumming, it doesn't make sense to waste CPU cycles on checksums when the use case doesn't require it. ext4 is a perfectly fine filesystem for most desktop users and a wide range of servers simply by not being ZFS. If you need CoW and checksumming, there is btrfs too, which is natively supported on Linux and IMO fine if you're only on a single disk or use md or LVM for RAID. It's quite amusing tbh when people think that a filesystem needs to do everything.

tjoff 9y ago
Checksumming is free on todays CPUs, you could use your argument agains filesystem journaling as well. But for some reason you are not proposing using ext2? Why not? Why waste clock cycles on a journal when you might not need it? This truly is a non-issue but in very exceptional circumstances, and in such you don't use a general purpose filesystem anyway. Checksumming and snapshots etc. give substantial game-changing improvements to the whole system. More than a journaling filesystem ever did. They also provide means as to vastly simplify other tasks. Roll-back last system-wide update? Done. Backup consistency? Free and vastly simplified. I disagree, these are properties that are expected of any decent general purpose filesystem. If you fall into a niche that doesn't, fine, use one of the many alternatives. But this should be the default for all user-facing machines as well as servers.
- tscs37 9y ago
  Checksumming is rather not free on today's CPUs unless you use something like xxhash which operates close to RAM speed. Journaling is rather useful because especially consumer desktop computers tend to be mistreated and crash a lot. The physical journal in ext4 prevents the worst when stuff gets hairy. Additionally, it does not effectively delete data like ZFS (or rather, make it very very hard to access data) just because a bit was corrupted. I think a user will value their family pictures more than some random text file suddenly having another character somewhere, they will loose data with a checksumming FS on a normal machine. There is no second drive to pull a good copy of the data from. There is one. And you just killed the last picture of grandma. Congrats. >Why waste clock cycles on a journal when you might not need it? Journaling wastes mostly bandwidth and IO, not much CPU and clock cycles. >Roll-back last system-wide update? Done. Backup consistency? Free and vastly simplified. That is indeed something you can do with a CoW system but the tools to achieve this are in my experience not fully mature yet. The same tasks can also be achieved without snapshots at all, NixOS seems to be doing fine. >Backup consistency? Free and vastly simplified. Snapshots are not backups. If you pretend they are you will loose all your data. Snapshots are snapshots. Nothing more and nothing less. Not backups. >But this should be the default for all user-facing machines as well as servers. I've not stated they shouldn't be default, rather, that the end user does not necessarily need them. Though, ZFS is doing a rather poor job on all these tasks, especially disk management on ZFS is a hassle I wouldn't have a typical end user face. I think Bcache FS is a much cleaner and better design than ZFS, which is rather unflexible. But again, I don't see the immediate need for every end user machine to have a checksumming and CoW filesystem, people have been doing fine for years without them and I doubt there is chaos and fire everywhere as you seem to imply. It could be a little bit better but ext4 is a perfectly fine filesystem for production data. If you rely on checksumming and snapshots for your data safety, you're in for a bad time. Proper backups, not snapshots, will beat snapshots any day. Checksums make data inaccessible, a rather bad choice for a user machine. Whoops, I guess those family photos are totally unusuable because 1 bit has corrupted.
  - heinrich5991 9y ago
    >>Backup consistency? Free and vastly simplified. >Snapshots are not backups. If you pretend they are you will loose all your data. The good thing about snapshots it that you can use them as a consistent base for your backup: If you naively copy files over from a running system, you'll get inconsistent data, but if you copy it over from a snapshot, it'll work without taking the system down.
    - tscs37 9y ago
      That is fairly obvious, a lot of systems provide snapshot functionality irrelevant of the file system. If you still need snapshots, there are several methods without needing CoW Filesystems, notably LVM and dattobd (both of which I have utilized in the past) Also keep in mind that a pure snapshot from a running system will behave like a system that just crashed, a half written file won't complete the write magically because you have snapshot. If you don't tell postgresql that you're doing snapshots or backups, both will likely corrupt data, doesn't give you anything in that case. Same for desktop systems. You make a snapshot of the runnign system and suddenly the browser only finds corrupted settings because a write wasn't complete until after the snapshot.
      - Dylan16807 9y ago
        If postgresql corrupts your data upon sudden halt you have far bigger problems to worry about. And as far as I know most software acts responsibly with files, either writing out the new version to a temp file or using sqlite. Both of which are atomic-snapshot-safe.
        tscs37 9y ago
        You might still loose partial datasets (especially when the database is not 100% perfectly constrained) and not all software handles such sudden halts perfectly. Making a snapshot of any working dataset of any software without the data corrupting or loosing parts of the working set is a pipedream.
    - 9y ago

by tscs37 9y ago

tjoff 9y ago

Checksumming is free on todays CPUs, you could use your argument agains filesystem journaling as well. But for some reason you are not proposing using ext2? Why not? Why waste clock cycles on a journal when you might not need it? This truly is a non-issue but in very exceptional circumstances, and in such you don't use a general purpose filesystem anyway. Checksumming and snapshots etc. give substantial game-changing improvements to the whole system. More than a journaling filesystem ever did. They also provide means as to vastly simplify other tasks. Roll-back last system-wide update? Done. Backup consistency? Free and vastly simplified. I disagree, these are properties that are expected of any decent general purpose filesystem. If you fall into a niche that doesn't, fine, use one of the many alternatives. But this should be the default for all user-facing machines as well as servers.

tscs37 9y ago
Checksumming is rather not free on today's CPUs unless you use something like xxhash which operates close to RAM speed. Journaling is rather useful because especially consumer desktop computers tend to be mistreated and crash a lot. The physical journal in ext4 prevents the worst when stuff gets hairy. Additionally, it does not effectively delete data like ZFS (or rather, make it very very hard to access data) just because a bit was corrupted. I think a user will value their family pictures more than some random text file suddenly having another character somewhere, they will loose data with a checksumming FS on a normal machine. There is no second drive to pull a good copy of the data from. There is one. And you just killed the last picture of grandma. Congrats. >Why waste clock cycles on a journal when you might not need it? Journaling wastes mostly bandwidth and IO, not much CPU and clock cycles. >Roll-back last system-wide update? Done. Backup consistency? Free and vastly simplified. That is indeed something you can do with a CoW system but the tools to achieve this are in my experience not fully mature yet. The same tasks can also be achieved without snapshots at all, NixOS seems to be doing fine. >Backup consistency? Free and vastly simplified. Snapshots are not backups. If you pretend they are you will loose all your data. Snapshots are snapshots. Nothing more and nothing less. Not backups. >But this should be the default for all user-facing machines as well as servers. I've not stated they shouldn't be default, rather, that the end user does not necessarily need them. Though, ZFS is doing a rather poor job on all these tasks, especially disk management on ZFS is a hassle I wouldn't have a typical end user face. I think Bcache FS is a much cleaner and better design than ZFS, which is rather unflexible. But again, I don't see the immediate need for every end user machine to have a checksumming and CoW filesystem, people have been doing fine for years without them and I doubt there is chaos and fire everywhere as you seem to imply. It could be a little bit better but ext4 is a perfectly fine filesystem for production data. If you rely on checksumming and snapshots for your data safety, you're in for a bad time. Proper backups, not snapshots, will beat snapshots any day. Checksums make data inaccessible, a rather bad choice for a user machine. Whoops, I guess those family photos are totally unusuable because 1 bit has corrupted.
- heinrich5991 9y ago
  >>Backup consistency? Free and vastly simplified. >Snapshots are not backups. If you pretend they are you will loose all your data. The good thing about snapshots it that you can use them as a consistent base for your backup: If you naively copy files over from a running system, you'll get inconsistent data, but if you copy it over from a snapshot, it'll work without taking the system down.
  - tscs37 9y ago
    That is fairly obvious, a lot of systems provide snapshot functionality irrelevant of the file system. If you still need snapshots, there are several methods without needing CoW Filesystems, notably LVM and dattobd (both of which I have utilized in the past) Also keep in mind that a pure snapshot from a running system will behave like a system that just crashed, a half written file won't complete the write magically because you have snapshot. If you don't tell postgresql that you're doing snapshots or backups, both will likely corrupt data, doesn't give you anything in that case. Same for desktop systems. You make a snapshot of the runnign system and suddenly the browser only finds corrupted settings because a write wasn't complete until after the snapshot.
    - Dylan16807 9y ago
      If postgresql corrupts your data upon sudden halt you have far bigger problems to worry about. And as far as I know most software acts responsibly with files, either writing out the new version to a temp file or using sqlite. Both of which are atomic-snapshot-safe.
      - tscs37 9y ago
        You might still loose partial datasets (especially when the database is not 100% perfectly constrained) and not all software handles such sudden halts perfectly. Making a snapshot of any working dataset of any software without the data corrupting or loosing parts of the working set is a pipedream.
  - 9y ago