4 ms·
What about data safety? Say, a hard drive got a bad sector, then a whole file is broken. This always stopped me from using system-wide compression. But maybe it
by wmu 9y ago
What about data safety? Say, a hard drive got a bad sector, then a whole file is broken. This always stopped me from using system-wide compression. But maybe it's not as bad as I suppose.
- zokier 9y agoIn theory you should get far better robustness by combined use of compression and FEC/ECC over storing content as-is. In practice this relies on the data being at least somewhat compressible, which might be an issue
- dom0 9y agoFS compression does not compress the whole file in one block.
- squarefoot 9y agoIn case of files primarily composed of redundant data, making 2 or more copies on the fly would still save a lot of space. Might be also useful for working in mixed SSD+HDD environments where less data would be written to the SSD and the HDD would hold copies of the files for safety.
- hultner 9y agoI've personally been running ZFS with LZ4, where checksums handle corruption. Another upside is that LZ4 is fast enough to actually improve I/O throughput, since overhead is negligible. On my jail/container/vm datasets I've seen space reduction upwards 80%, with backup-datasets I've seen around 30% reduction. Another possibility is data deduplication, which is useful if you have plenty of RAM. You can deduplicate across files on block level. However I have only used it sporadically due to the high ram usage.