3 ms·
It’s not so much about IO rate but access times. If compression and decompression speed are critical, lz4. Zstd for pretty much everything else. There are edg
by jorvi 3y ago
It’s not so much about IO rate but access times.
If compression and decompression speed are critical, lz4. Zstd for pretty much everything else.
There are edge cases where compression time doesn’t matter but decompression time does. This used to be the case for game torrents back in the old days, and UHARC was used for that to great effect. Not sure what the current king is for that purpose.
- lifthrasiir 3y agoBrotli has a higher potential for the compression ratio due to its internal structure, so that edge case would be better served by Brotli than by Zstandard.
- repiret 3y agoOne of the systems I maintain at work uses enormous, highly compressible, text files, which need to be compressed once and decompressed many times. Decompression speed isn't critical, it just needs to keep up with processing the decompressed data to avoid being the bottleneck. We optimize primarily for compression ratio. For that system, we haven't found something that beats `xz -9`.
- sodality2 3y agoSomething like FreeArc would be used nowadays for repacks, typically - there's gains to be made by tailoring your procedure depending on the type of game asset, and some of those decisions won't apply to general compression. FreeArc Next does actually use zstd as above but it also does a lot of tricks with compression, dictionaries, etc while taking much longer to process. As an example, looking at FitGirl's COD:BO3 repack, 180GB->42.4GB entirely losslessly. Not sure how regular compression would fare on the original game, though.