4 ms·

This looks very similar to attic, anyone able and willing to compare them for pros and cons?

by ncza 12y ago

leni536 12y ago
I would throw bup[1] into the competition too. One pro over zbackup is the easier retrieval of single files. Bup uses the same rolling checksum method for dedupliation inspired by rsync but it uses git pack files to store the data. Some git functionalities are reimplemented to work with large number of files and large files. [1] https://github.com/bup/bup https://github.com/bup/bup edit: One huge con is that you can't prune old backup data. It's really early in development but it looks promising.
jewel 12y ago
It so happens that I found zbackup in the list of apt packages just yesterday. It doesn't have network support like attic, so you have to transport the entire repository of data to the backup server in order to make a backup. In practice that's impractical for anything big. You can use rsync to do the transfer to a local clone of the data on the backup server but then the whole thing still needs to be ingested each time. It has a rolling checksum splitter just like attic, which is unbelievably effective way to split files into chunks for deduplication. It'll work really well for database dumps, which is something that fixed-size chunking fails at miserably. A big advantage over bup is that you can remove old backups. Other backup software worth looking at is rdiff-backup, duplicity, burp, and obnam, all of which are in apt.
- leni536 12y ago
  You can easily use netcat to pipe tar over the network directly to zbackup. If you want encryption over the network it's even easier: tar -c stuff/ | ssh user@example.com zbackup backup /my/backup/repo/backups/backup-`date '+%Y-%m-%d'`
  - jewel 12y ago
    I should have been more clear. The problem is that zbackup won't work for large data sets. For example, I used to use rdiff-backup for a samba filesystem at work that had a few terabytes of files on it. The problem was that whenever someone would rename a directory all of the data in its subdirectories would have to be backed up again. Backup software that deduplicates solves this problem wonderfully, since it doesn't make a difference if the files have been moved or not. The problem with your approach is that I can't send several terabytes of data offsite every night. I could use the rsync trick I mentioned, but now I've got to store two copies of the data on the backup server.
    - Am1GO 12y ago
      Wrong. You could keep in sync only "index" and "backups" directories on your backup client with server, you don't have to keep "bundles" everywhere.

This looks very similar to attic, anyone able and willing to compare them for pros and cons?

by ncza 12y ago

This looks very similar to attic, anyone able and willing to compare them for pros and cons?

I would throw bup[1] into the competition too. One pro over zbackup is the easier retrieval of single files. Bup uses the same rolling checksum method for dedupliation inspired by rsync but it uses git pack files to store the data. Some git functionalities are reimplemented to work with large number of files and large files. [1] https://github.com/bup/bup https://github.com/bup/bup edit: One huge con is that you can't prune old backup data. It's really early in development but it looks promising.

jewel 12y ago

It so happens that I found zbackup in the list of apt packages just yesterday. It doesn't have network support like attic, so you have to transport the entire repository of data to the backup server in order to make a backup. In practice that's impractical for anything big. You can use rsync to do the transfer to a local clone of the data on the backup server but then the whole thing still needs to be ingested each time. It has a rolling checksum splitter just like attic, which is unbelievably effective way to split files into chunks for deduplication. It'll work really well for database dumps, which is something that fixed-size chunking fails at miserably. A big advantage over bup is that you can remove old backups. Other backup software worth looking at is rdiff-backup, duplicity, burp, and obnam, all of which are in apt.

leni536 12y ago
You can easily use netcat to pipe tar over the network directly to zbackup. If you want encryption over the network it's even easier: tar -c stuff/ | ssh user@example.com zbackup backup /my/backup/repo/backups/backup-`date '+%Y-%m-%d'`
- jewel 12y ago
  I should have been more clear. The problem is that zbackup won't work for large data sets. For example, I used to use rdiff-backup for a samba filesystem at work that had a few terabytes of files on it. The problem was that whenever someone would rename a directory all of the data in its subdirectories would have to be backed up again. Backup software that deduplicates solves this problem wonderfully, since it doesn't make a difference if the files have been moved or not. The problem with your approach is that I can't send several terabytes of data offsite every night. I could use the rsync trick I mentioned, but now I've got to store two copies of the data on the backup server.
  - Am1GO 12y ago
    Wrong. You could keep in sync only "index" and "backups" directories on your backup client with server, you don't have to keep "bundles" everywhere.