3 ms·

I am still somewhat suprised there are still not relatively simple filesystems that don't do this. Make it the one feature it does and does properly. Adding ent

I am still somewhat suprised there are still not relatively simple filesystems that don't do this. Make it the one feature it does and does properly. Adding enterprise features will just increase development time (and cost) and add bugs. Basically an online archival filesystem, that's how most people use their home computers. Requirements: 1. Checksumming on all files 2. Minimise assumptions of ram correctness 3. Disc replication (soft raid 1) Out of scope: * Snapshotting * Subvolumes * Deduplication As much as I love subvolumes and snapshotting, I feel like CoW probably adds too much complexity to make it (easily) reliable. Honestly I don't know why turning off CoW on BTRFS disables checksumming, so if anyone can shed some light as to why this is, feel free to point out that my requirements are far more complex than I think.

blattimwind 9y ago
> I don't know why turning off CoW on BTRFS disables checksumming Because in-place updates means you need to journal data writes, otherwise you'll get checksum errors for aborted writes. (Which classic file systems don't catch) The recent (~5 years old) batch of block-deduplicating backup tools (attic, bup, borg, restic etc.) would fit the bill to some extent, but all of them were/are more bug-laden than the file systems discussed so far.
boomboomsubban 9y ago
>As much as I love subvolumes and snapshotting, I feel like CoW probably adds too much complexity to make it (easily) reliable. Despite this bug, ZFS still seems reliable and does everything you ask. And any reliability you feel is gained by dropped CoW is offset by the reliability lost by being susceptible to power outages corrupting data.
zzzcpan 9y ago
Checksumming is like a myth on HN already. No, it's not useful on its own to the user or to the filesystem, but it is useful to the filesystem if it can transparently heal the data. Whole disk replication doesn't address your reliability concerns as much as you think either, doesn't work all that well in general and is a burden to maintain. Doing it properly requires treating the filesystem as a distributed system where disks can join and leave, where everything is sharded, where rebalancing, self-healing, syncing, resyncing is all automatic and blazing fast. And so on. Nowadays most of the work in storage systems is in distributed systems, and local filesystems are treated as just another unreliable layer. Hope this gives you some ideas why there are no filesystems like that.
- Arbalest 9y ago
  Not particularly. I'm thinking for the home user who just copies the photos from their camera to their home computer. Distributed systems would be great, were it not for the fact that by and large, home connections have abysmal upload speeds.
ryao 9y ago
In-place writes invalidate checksums. It is theoretically possible for an in-place filesystem to have data checksums, but maintaining them would require reading the entire block to regenerate the checksum on every write (I consider reads from cache to be reads). That largely defeats the purpose of doing operations in-place, which is to avoid that overhead. Maybe it would make more sense on btrfs (because it’s metadata structures can become horribly unbalanced) than it usually would on a convention in-place filesystem. However, the code would need to do gymnastics to achieve that without risking subtle reliability issues. That is something that I would not expect many filesystem developers to want to maintain. The btrfs codebase is complex enough (more so than ZFS despite having fewer features) without adding that. By the way, partial writes from power failures are impossible to correct on in place filesystem designs without writing everything twice. CoW has performance penalties, but not so much as in-place designs when it comes to ensuring integrity. The complexity of doing CoW is also not that bad. It is how virtual memory works in every modern computer system, minus some rare embedded systems and ancient/unikernel designs that lack it. Doing CoW in storage is not a particularly strange thing.

blattimwind 9y ago

> I don't know why turning off CoW on BTRFS disables checksumming Because in-place updates means you need to journal data writes, otherwise you'll get checksum errors for aborted writes. (Which classic file systems don't catch) The recent (~5 years old) batch of block-deduplicating backup tools (attic, bup, borg, restic etc.) would fit the bill to some extent, but all of them were/are more bug-laden than the file systems discussed so far.

boomboomsubban 9y ago

>As much as I love subvolumes and snapshotting, I feel like CoW probably adds too much complexity to make it (easily) reliable. Despite this bug, ZFS still seems reliable and does everything you ask. And any reliability you feel is gained by dropped CoW is offset by the reliability lost by being susceptible to power outages corrupting data.

zzzcpan 9y ago

Checksumming is like a myth on HN already. No, it's not useful on its own to the user or to the filesystem, but it is useful to the filesystem if it can transparently heal the data. Whole disk replication doesn't address your reliability concerns as much as you think either, doesn't work all that well in general and is a burden to maintain. Doing it properly requires treating the filesystem as a distributed system where disks can join and leave, where everything is sharded, where rebalancing, self-healing, syncing, resyncing is all automatic and blazing fast. And so on. Nowadays most of the work in storage systems is in distributed systems, and local filesystems are treated as just another unreliable layer. Hope this gives you some ideas why there are no filesystems like that.

Arbalest 9y ago
Not particularly. I'm thinking for the home user who just copies the photos from their camera to their home computer. Distributed systems would be great, were it not for the fact that by and large, home connections have abysmal upload speeds.

ryao 9y ago

In-place writes invalidate checksums. It is theoretically possible for an in-place filesystem to have data checksums, but maintaining them would require reading the entire block to regenerate the checksum on every write (I consider reads from cache to be reads). That largely defeats the purpose of doing operations in-place, which is to avoid that overhead. Maybe it would make more sense on btrfs (because it’s metadata structures can become horribly unbalanced) than it usually would on a convention in-place filesystem. However, the code would need to do gymnastics to achieve that without risking subtle reliability issues. That is something that I would not expect many filesystem developers to want to maintain. The btrfs codebase is complex enough (more so than ZFS despite having fewer features) without adding that. By the way, partial writes from power failures are impossible to correct on in place filesystem designs without writing everything twice. CoW has performance penalties, but not so much as in-place designs when it comes to ensuring integrity. The complexity of doing CoW is also not that bad. It is how virtual memory works in every modern computer system, minus some rare embedded systems and ancient/unikernel designs that lack it. Doing CoW in storage is not a particularly strange thing.