3 ms·
Couple of comments. Firstly, you are talking about highly redundant information when referencing VM images (e.g. the C drive on all Windows Serer images will be
by abrookewood 2y ago
Couple of comments. Firstly, you are talking about highly redundant information when referencing VM images (e.g. the C drive on all Windows Serer images will be virtually identical), whereas he was using his own laptop contents as an example.
Secondly, I think you are conflating two different features: compression & de-duplication. In ZFS you can have compression turned on (almost always worth it) for a pool, but still have de-duplication disabled.
- UltraSane 2y agoFair point. My experience is with enterprise storage arrays and I have always used dedupe/compression at the same time. Dedupe is going to be a lot less useful on single computers. I consider dedupe/compression to be two different forms of the same thing. compression reduces short range duplication while deduplication reduces long range duplication of data.
- abrookewood 2y agoYeah agreed, very closely related - even more so on ZFS where the compression (AFAIK) is on a block level rather than a file level.
- E39M5S62 2y agoZFS compression is for sure at the block level - it's fully transparent to the userland tools.
- lazide 2y agoIt could be at a file level and still transparent to user land tools, FYI. Depending on what you mean by ‘file level’, I guess.
- UltraSane 2y agoWindows NTFS has transparent file level compression that works quite well.
- Dylan16807 2y agoI don't know how much I agree with that. The old kind of NTFS compression from 1993 is completely transparent, but it uses a weak algorithm and processes each 64KB of file completely independently. It also fragments files to hell and back. The new kind from Windows 10 has a better algorithm and can have up to 2MB of context, which is quite reasonable. But it's not transparent to writes, only to reads. You have to manually apply it and if anything writes to the file it decompresses. I've gotten okay use out of both in certain directories, with the latter being better despite the annoyances, but I think they both have a lot of missed potential compared to how ZFS and BTRFS handle compression.
- UltraSane 2y agoI'm talking about the "Compress contents to save disk space" option in the Advanced Attributes. It makes the file blue. I enable it on all .txt log files because it is so effective and completely transparent. It compresses a 25MB Google Drive log file to 8MB
- Dylan16807 2y agoThat's the old kind. It's useful, but if they updated it it could get significantly better ratios and have less impact on performance.