5 ms·
Why not use BTRFS with file deduplication and transparent compression (zstd specifically)?
by hachari 6y ago
Why not use BTRFS with file deduplication and transparent compression (zstd specifically)?
- TimTheTinker 6y agoThis is a read-only file system, so it’s able to exploit certain properties of that—- locating similar files next to each other, for example.
- hachari 6y agoI don’t see how read only helps at all. Btrfs can dedupe at the block level.
- _flux 6y agoConsider how much of work in btrfs is done to just handle the case of modifying existing files—or reducing file system size.. It is basically the reason it uses b-trees. It's in the name! For example, when dedupping in block level it needs to know (right?) how many times a block is being used, so it can be collected when it runs out of uses. ISO9660 can also express dedupped (hardlinked) files with the Rock Ridge extensions. I don't know but I'm wondering it could even do block-level dedupping if the generating program abused the format a bit..
- TimTheTinker 6y agoSure, but it sounds like block deduping is only one of several optimizations that DwarFS is able to take advantage of because it’s a read-only FS.
- fsiefken 6y agoA compression benchmark of both filesystems would be of interest in this regard (lzo, zstd and zlib), both read speed and compression wise
- tutfbhuf 6y agoIs Btrfs stable yet?
- Diederich 6y agoYup: https://lwn.net/Articles/824855/ https://lwn.net/Articles/824855/
- hachari 6y agoBtrfs is default in Fedora 33.
- tutfbhuf 6y agoFedora also has Wayland as default. If Debian had Btrfs as default, that would be an argument.
- Arnavion 6y agoIt's also been the default for the root filesystem in openSUSE since 2012. It also integrates with the package manager - zypper creates pre- and post- snapshots when installing updates, and the snapper utility makes it easy to roll back if your system is busted after installing an update. It's saved my bacon on multiple occasions. (btrfs also used to be the default for /home, though that changed at some point. When I made a new install last year the installer suggested xfs by default.)
- packetlost 6y agoFor most use-cases, yes. But not if you plan on using raid5+ [0] [0]: https://btrfs.wiki.kernel.org/index.php/Gotchas#Parity_RAID https://btrfs.wiki.kernel.org/index.php/Gotchas#Parity_RAID My own experience indicates it's brittle to power failures and will corrupt in annoying ways in the event of a power failure (or hard reboot) as of ~1 year ago.
- hachari 6y agoYear isnt as useful as kernel version.