3 ms·
What compression was used before Arq 5? lz4 is super fast, but not particularly space-efficient compared to some slower compression algorithms. Since Arq custom
by enoch_r 11y ago
What compression was used before Arq 5? lz4 is super fast, but not particularly space-efficient compared to some slower compression algorithms. Since Arq customers are the ones paying the storage bills, this doesn't seem like an entirely costless decision--Arq is now faster, but you should expect your storage bills to go up a bit due to the lower compression.
I use Borg backup with lz4 compression, so I definitely don't think this is the wrong decision, just something to keep in mind (and, it does seem like something that could and maybe should be user-configurable).
- skrause 11y agoArq's data format description says that the other supported compression type is gzip, which I assume was used before: https://www.arqbackup.com/arq_data_format.txt https://www.arqbackup.com/arq_data_format.txt
- lobster_johnson 11y agoI'm also curious about the choice of LZ4. It's fast, but its compression is pretty awful. For example, on simple, regular text (JSON) files I'm seeing about 45% worse compression than plain gzip (default compression level). I hope at least it's using the highest compression level, but I've found that "lz4 -9" is about as slow as "gzip -6", still with worse compression. I'd be happier if the choice of compression was dependent on the size of files. The larger a file, the more you gain from compression. Does Arq skip compression for already-compressed files (.gz, .xz, .bz2 etc.)?
- xref 11y agoCurious decision, as lz4's terrible compression ratio burns you both in monthly storage fees, as well as Glacier (or other service) restore fees. Most of my backup set is pictures and compressed files, so maybe the lz4 decision was because they figured the majority of their customers fell in that category where no compression algorithm is going to help?