5 ms·
Another major problem is that it does not support compression: all files are stored uncompressed on the backup server (even though they are deduplicated) (sic).
by Erlich_Bachman 7y ago
Another major problem is that it does not support compression: all files are stored uncompressed on the backup server (even though they are deduplicated) (sic).
- mike-cardwell 7y agoYou know, I actually assumed it had compression. It looks like it will get it eventually though, so I'm not too worried. That said, I can get some benefits by compressing some of my backups myself. For example, I currently backup my postgres dbs with cron jobs like: $ pg_dump dbname | restic backup --stdin --stdin-filename dbname.dump --tag dbname -q With restoring as simple as: $ restic dump latest --tag dbname dbname.dump | psql dbname I should probably stick a gzip/gunzip between those two commands.
- adrianmonk 7y agoIf you stick a cheese up in there, restic will probably lose some of its ability to deduplicate. If it does compression, then it's not an opaque process to it. But if you do it, you are essentially obfuscating the structure of the data.
- mike-cardwell 7y agoGood point. I'll stay away from cheesing up my commands and wait for it to be built in.
- mynegation 7y agoWhat is "cheesing up"?
- adrianmonk 7y agoIt's what Google speech-to-text thinks I meant when I said "gzip" into my phone.
- sliken 7y agoHa, I was thinking there's a new gzip competitor I needed to track down called cheese.
- tatersolid 7y agogzip with the —rsyncable flag will still allow for deduplication. It uses a content-aware rolling checksum and resets the compression dictionary whenever that checksum has a certain number of zeros. The result is a valid gzip file with a few percent less compression, but friendly to deduplication (which is what rsync does).
- BurningCycles 7y agoYes, that is the one thing that keeps me from using Restic as my backup solution. Unfortunately it does not seem like it's a prioritised feature given that it has been planned since late 2014.
- eps 7y agoIf this is a concern, why not delegate this to the file system level - just stick backups on a compressed volume and be done with it, no?
- Erlich_Bachman 7y agoNo. You cannot stick a compression step before restic, because it will mess up the deduplication - the chunker will not find the same chunks, and will not be able to find out which files have same contents (not as easily at least). And you cannot stick it after restic, compressing the backup archives (or storing them on a compressing filesystem), because those are encrypted, and any encrypted file has high entropy and does not compress. (It doesn't help that restic makes the encrypotion mandatory). So no, compression really needs to be an integral step in a backup system. Restic is definitely lacking in this regard.
- eps 7y agoAh, you are right. I forgot it was encrypting the backups.