7 ms·

A performance comparison of Duplicacy, restic, Attic, and duplicity

46 points by acrosync 9y ago

atonse 9y ago
Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). That would beat having to install something like Backblaze on every family member's machines. Cloud backup is great but it's always better to have a local (LAN) copy and then an off-site copy.
- robotmay 9y ago
  I use borg (attic fork) backing up to rsync.net for my home server. All my machines back up locally to that machine (mostly using SyncThing), then it backs itself up every hour or so. It's not perfect but it does work really rather well. Borg is really nice, and rsync.net is of that variety of service that are always my favourites: it does one thing very well. Also they offer a discount if you use borg or attic (possibly others) as they turn off their ZFS snapshot system and assume your software handles that.
  - tombrossman 9y ago
    How much data and what's your monthly cost? I have a 2TB storage VPS for under $10/month. rsync.net looks very good but is possibly total overkill for my needs. Definitely don't need >1 snapshot/day as that's what my hourly local backup is for.
    - robotmay 9y ago
      It's a bit more expensive than most options but the price does vary by how much of your data is duplicated. At the mo I have 1TB of data being backed up but it squashes down to under 550GB with borg. Monthly price for me is about $17 (paid yearly though). I'm happy with the extra price for my use, but yeah it's not the best option if you have a lot of unduplicated data. If you're interested, here's the link to the borg/attic pricing page: http://www.rsync.net/products/attic.html http://www.rsync.net/products/attic.html
    - GordonS 9y ago
      Can I ask where you got the VPS? That's a really great price for 2TB!
      - tombrossman 9y ago
        At present I'm using a provider in Lithuania called time4vps. Overall the service is good (assuming you are connecting from Europe) but to use their website I have to disable my ad-blocking & privacy add-ons, which I don't have to do on other providers' sites. Not sure why that is. I'll probably try Delimiter once they start offering service in London, as they also have some similar low cost + high storage plans.
- tombrossman 9y ago
  > Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). Do you mean backup to the NAS or for backing up the NAS itself to a third location? Either way, there's no need for an either/or approach. Just do both. I've tried multiple backup applications and many support local and remote backup options as standard. I'm using Borg and Back In Time and with Borg it's just a second cronjob with nearly identical scripts for the off-site backup over SSH. With Back In Time I was using the AWS CLI to push the backups to S3, but I found a better deal with cheaper storage. I have about 200GB data in total but want a 1TB archive available online for older backup sets. Side note - OP's review is helpful but Borg Backup is oddly listed under 'Attic', which it was forked from years ago. Don't bother looking for Attic if you are comparing backup tools available today.
- 2bluesc 9y ago
  > Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). I use borgbackup to back-up my stuff locally with rclone to mirror the borg repository in the cloud (personal Google Drive in my case) and have also experimented with running borgbackup to an offsite Raspberry Pi.
- flipbrad 9y ago
  Duplicity is pretty easy to use with Backblaze B2 as a cloud storage backend - that's what I use to backup my NAS.
whois 9y ago
Has anyone run these on their computer? As they noted, they are the author of Duplicacy.
- acrosync 9y ago
  Author of Duplicacy here. I would like to see other's results too.
- stevekemp 9y ago
  I used to use attic to backup my personal Debian systems, since upgrading to the new stable (Stretch) release it is no longer available, so I switched to borg. I had to juggle a few things around, but it works well. As does obnam, which I use in a couple of other places too.
dabeeeenster 9y ago
Best thing about restic is not having to spend 45 minutes downloading and manually bullshitting around with python dependencies and libraries. What Go is good at IMO.
- rkrzr 9y ago
  `attic` and `borg` are just one `sudo apt install attic` or `sudo apt install borgbackup` away. No "bullshitting around with python dependencies" required. It looks like they also have packages for other platforms: https://github.com/borgbackup/borg/releases https://github.com/borgbackup/borg/releases
  - dabeeeenster 9y ago
    I was talking mainly about duplicity which has always been like pulling teeth.
brunoqc 9y ago
It's a shame Duplicacy is not free software.
- acrosync 9y ago
  Author of Duplicacy here. To personal users, it is free software.
  - hannob 9y ago
    The term "free software" has a very specific meaning [1] and there is no such thing as "free software, but only for personal users". [1] https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
  - yorwba 9y ago
    The unambiguous term would be "gratis" or "free of charge". Your software is not "free software" in the "libre" or "free to modify and share" sense.
  - comice 9y ago
    It doesn't meet the free software foundation's definition of free software, which I believe was the original point. It lacks the freedom to run the program as you wish, for any purpose (freedom 0). https://en.wikipedia.org/wiki/The_Free_Software_Definition#The_definition_and_the_Four_Freedoms https://en.wikipedia.org/wiki/The_Free_Software_Definition#T... https://github.com/gilbertchen/duplicacy/blob/master/LICENSE.md https://github.com/gilbertchen/duplicacy/blob/master/LICENSE...
    - _wxn8 9y ago
      The FSF's definition of "Free Software" doesn't necessarily match the English language's definition of "free software". Since we're speaking English, I think it's reasonable to assume the latter meaning, like most reasonable people who haven't encountered the FSF would. I really wish people would capitalize things like this. "This is not Free Software" would make the sentence unambiguous. You can't just go around redefining the English language willy nilly and expect people to play along. Languages do evolve naturally but this isn't natural. This is an organization trying to influence the language to advance an agenda (though I believe it to be a worthy agenda, it's still an agenda).
crdoconnor 9y ago
Performance bothers me an order of magnitude less than the potential for obscure bugs which could lose me data.
2bluesc 9y ago
The results don't look very complete seeing as how attic was abandoned over two years ago. The master attic branch[0] has 600 commits. The fork of attic, borg, has over 4000 commits [1] suggesting a significant amount of work has been done to improve it. It seems odd for the author to compare it to something abandoned (and thankfully reborn as borg) and ignore what has happened in two years. Would love to see similar tests run against borg. [0] https://github.com/jborg/attic https://github.com/jborg/attic [1] https://github.com/borgbackup/borg https://github.com/borgbackup/borg
- zokier 9y ago
  The version row for attic says "BorgBackup 1.1.0b6"
- acrosync 9y ago
  Author here. The experiments actually ran BorgBackup 1.1.0b6 as you can see from the Setup section. We liked to call it Attic out of the respect to the original Attic author.
  - tombrossman 9y ago
    You realize the name Borg comes from the Attic author, Jonas Borgström, right? No one else calls it Attic as the two are different projects.
    - acrosync 9y ago
      I noticed that, but didn't know if Borg has another meaning. I can understand why they forked the project, but in my opinion a name that makes the origin more obvious would have been better.
      - dom0 9y ago
        "Borg" was chosen, because it emphasizes collaborative development — and because someone is a Star Trek fan ;)
luxpir 9y ago
Could obnam be added? I use it and with a few tweaks it's very respectable in terms of performance. Also has some good integrity checks.
ams6110 9y ago
> duplicity has a serious flaw in its incremental model -- the user has to decide whether to perform a full backup or an incremental backup on each run. That is because while an incremental backup saves a lot of storage space, it is also dependent on previous backups due to the design of duplicity, making it impossible to delete any single backup on a long chain of dependent backups. So there is always a dilemma of how often to perform a full backup for duplicity users. Yes and no. Duplicity has the "--full-if-older-than" option so you can do incrementals normally, but if your previous full is older than whatever interval you define, it will do a full backup, without changing the command line. So that can be e.g. in a cron job.
- willvarfar 9y ago
  Classic source control had this problem. The clever trick is to reencode the previous most-recent backup as a delta from the current state, and do a full-backup of the current state, rather than encoding each new backup as a delta from the previous state (which becomes slower and slower to compute, the more previous states you have). Problem solved :)
  - tatersolid 9y ago
    That's really expensive when your previous backup is on cloud storage
    - willvarfar 9y ago
      To compute a delta, you need the previous version. If this previous version is computed from a single file - the previous snapshot - then that's actually less data and effort than if its computed by taking an old snapshot and replaying all the deltas upto the current time.
beagle3 9y ago
i think bup does support concurrent access; the concurrent deduplication granularity is a backup set, so if two identical computers are backed up for the first time at exactly the same time you will not get deduplication - but that's inherently a dining philosopher kind of problem Also, recent bup versions allow delete. No encryption IiRC, but you can examine it with git tools which is a feature on its own.
AdmiralAsshat 9y ago
I would've been curious to see how BackinTime stacks up. Duplicity/deja-dup (a GNOME frontend for duplicity) is pre-installed on most GNOME-based DE's, which makes it convenient for end-users, but I found the limitation of only being able to configure a single backup destination to be too limiting. By contrast, BiT supported multiple destinations and profiles, meaning I could have one local, one off-site, one "Personal Data" backup, one "System" backup to fall back if an OS update fails, etc. Its configuration options were much more attractive.
e12e 9y ago
It's a little odd to not benchmark backup over the network - a backup taken to the same physical disk as the source of the data isn't very useful - for that use-case taking a filesystem snapshot[s] would probably be faster and more useful. Perhaps in combination with a checksumming tool, like [c], or with a filesystem like ZFS. Also, it can be difficult in a lot of environments to sustain more than 100mbps write to a remote, off-site system - halving the stored data can be a much bigger win then. All that said, it's interesting to see that a) duplicity seems slow, and b) very consistent in terms of speed. I wonder if there's some low-hanging fruit for optimization there. Personally I've had some luck using backupninja[b] in combination with duplicity. It's one of the few Free alternatives that allow the backup-system to encrypt "one-way" - so that compromising the backup-system doesn't immediately give read access to encrypted backups. It's a bit complicated to set it up for separate encrypt-to and signing keys though :/ [s] Today I would probably recommend ZFS - but I've always wanted to give NILFS2 a real test, especially on solid-state disks: http://nilfs.osdn.jp/en/ http://nilfs.osdn.jp/en/ [c] https://github.com/Tripwire/tripwire-open-source https://github.com/Tripwire/tripwire-open-source http://aide.sourceforge.net/ http://aide.sourceforge.net/ https://github.com/integrit/integrit https://github.com/integrit/integrit (Speaking of projects that might be fun/useful to redo in a safe language like rust or go - it would appear this would be a prime example, btw. On the whole moving integrity to the fs, as with zfs might be the better option, though). [b] https://0xacab.org/riseuplabs/backupninja https://0xacab.org/riseuplabs/backupninja
- dom0 9y ago
  > All that said, it's interesting to see that a) duplicity seems slow, and b) very consistent in terms of speed. I wonder if there's some low-hanging fruit for optimization there. Duplicity is classic delta-backup. It always reads all files and calculates a delta to a different version of the file, hence the fairly consistent performance. Performance of deduplicating archivers is more difficult to predict.
amq 9y ago
If you are using duplicity, use this moment to check if it works. There is a serious bug which seems to affect systems with a lot of data: https://bugs.launchpad.net/duplicity/+bug/896728 https://bugs.launchpad.net/duplicity/+bug/896728
dpc_pw 9y ago
I wonder how rdedup would compare.
dom0 9y ago
My two (possibly biased, much like the author's) cents. - No network-based tests; e.g. a typical fast internet connection (say 100/40 or 50/20 MBit/s) with a few dozen ms latency to some server or cloud service. This is of course difficult because these tend to be bad on reproducibility. For a network-based test, not only time is interesting, but total RX/TX as well. - I'm really surprised at restic's performance. It uses far more CPU than Borg in almost all tests... and Borg is already notoriously inefficient in it's CPU usage when looking at object throughput (restic: "fast, efficient"?). I don't mean to bash, I'm just surprised. - restic's deduplication performance might hint at Rabin Fingerprints being worse than Buzhash, but there might be other issue(s) leading to this result. - Besides CPU time, memory (peak) usage would be interesting. > For instance, file hashes enable users to quickly identify which files in existing backups are changed. They also allow third-party tools to compare files on disks to those in the backups. To be fair, Borg can calculate a variety of file hashes (MD5, SHA1, SHA2, ...) on the fly with "borg list". There are "borg diff" (to compare two archives) and "borg mount -o versions" as well, though the latter is generally impractical for looking at a large number of archives. > Again, by not computing the file hash helped improve the performance, but at the risk of possible undetected data corruption. I can't deduce how the last part follows (", but..."). Care to explain?

46 points by acrosync 9y ago

Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). That would beat having to install something like Backblaze on every family member's machines. Cloud backup is great but it's always better to have a local (LAN) copy and then an off-site copy.

robotmay 9y ago
I use borg (attic fork) backing up to rsync.net for my home server. All my machines back up locally to that machine (mostly using SyncThing), then it backs itself up every hour or so. It's not perfect but it does work really rather well. Borg is really nice, and rsync.net is of that variety of service that are always my favourites: it does one thing very well. Also they offer a discount if you use borg or attic (possibly others) as they turn off their ZFS snapshot system and assume your software handles that.
- tombrossman 9y ago
  How much data and what's your monthly cost? I have a 2TB storage VPS for under $10/month. rsync.net looks very good but is possibly total overkill for my needs. Definitely don't need >1 snapshot/day as that's what my hourly local backup is for.
  - robotmay 9y ago
    It's a bit more expensive than most options but the price does vary by how much of your data is duplicated. At the mo I have 1TB of data being backed up but it squashes down to under 550GB with borg. Monthly price for me is about $17 (paid yearly though). I'm happy with the extra price for my use, but yeah it's not the best option if you have a lot of unduplicated data. If you're interested, here's the link to the borg/attic pricing page: http://www.rsync.net/products/attic.html http://www.rsync.net/products/attic.html
  - GordonS 9y ago
    Can I ask where you got the VPS? That's a really great price for 2TB!
    - tombrossman 9y ago
      At present I'm using a provider in Lithuania called time4vps. Overall the service is good (assuming you are connecting from Europe) but to use their website I have to disable my ad-blocking & privacy add-ons, which I don't have to do on other providers' sites. Not sure why that is. I'll probably try Delimiter once they start offering service in London, as they also have some similar low cost + high storage plans.
tombrossman 9y ago
> Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). Do you mean backup to the NAS or for backing up the NAS itself to a third location? Either way, there's no need for an either/or approach. Just do both. I've tried multiple backup applications and many support local and remote backup options as standard. I'm using Borg and Back In Time and with Borg it's just a second cronjob with nearly identical scripts for the off-site backup over SSH. With Back In Time I was using the AWS CLI to push the backups to S3, but I found a better deal with cheaper storage. I have about 200GB data in total but want a 1TB archive available online for older backup sets. Side note - OP's review is helpful but Borg Backup is oddly listed under 'Attic', which it was forked from years ago. Don't bother looking for Attic if you are comparing backup tools available today.
2bluesc 9y ago
> Is anyone using such tools as a backup for their NAS (and then using their NAS for time machine?). I use borgbackup to back-up my stuff locally with rclone to mirror the borg repository in the cloud (personal Google Drive in my case) and have also experimented with running borgbackup to an offsite Raspberry Pi.
flipbrad 9y ago
Duplicity is pretty easy to use with Backblaze B2 as a cloud storage backend - that's what I use to backup my NAS.

whois 9y ago

Has anyone run these on their computer? As they noted, they are the author of Duplicacy.

acrosync 9y ago
Author of Duplicacy here. I would like to see other's results too.
stevekemp 9y ago
I used to use attic to backup my personal Debian systems, since upgrading to the new stable (Stretch) release it is no longer available, so I switched to borg. I had to juggle a few things around, but it works well. As does obnam, which I use in a couple of other places too.

dabeeeenster 9y ago

Best thing about restic is not having to spend 45 minutes downloading and manually bullshitting around with python dependencies and libraries. What Go is good at IMO.

rkrzr 9y ago
`attic` and `borg` are just one `sudo apt install attic` or `sudo apt install borgbackup` away. No "bullshitting around with python dependencies" required. It looks like they also have packages for other platforms: https://github.com/borgbackup/borg/releases https://github.com/borgbackup/borg/releases
- dabeeeenster 9y ago
  I was talking mainly about duplicity which has always been like pulling teeth.

brunoqc 9y ago

It's a shame Duplicacy is not free software.

acrosync 9y ago
Author of Duplicacy here. To personal users, it is free software.
- hannob 9y ago
  The term "free software" has a very specific meaning [1] and there is no such thing as "free software, but only for personal users". [1] https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
- yorwba 9y ago
  The unambiguous term would be "gratis" or "free of charge". Your software is not "free software" in the "libre" or "free to modify and share" sense.
- comice 9y ago
  It doesn't meet the free software foundation's definition of free software, which I believe was the original point. It lacks the freedom to run the program as you wish, for any purpose (freedom 0). https://en.wikipedia.org/wiki/The_Free_Software_Definition#The_definition_and_the_Four_Freedoms https://en.wikipedia.org/wiki/The_Free_Software_Definition#T... https://github.com/gilbertchen/duplicacy/blob/master/LICENSE.md https://github.com/gilbertchen/duplicacy/blob/master/LICENSE...
  - _wxn8 9y ago
    The FSF's definition of "Free Software" doesn't necessarily match the English language's definition of "free software". Since we're speaking English, I think it's reasonable to assume the latter meaning, like most reasonable people who haven't encountered the FSF would. I really wish people would capitalize things like this. "This is not Free Software" would make the sentence unambiguous. You can't just go around redefining the English language willy nilly and expect people to play along. Languages do evolve naturally but this isn't natural. This is an organization trying to influence the language to advance an agenda (though I believe it to be a worthy agenda, it's still an agenda).

crdoconnor 9y ago

Performance bothers me an order of magnitude less than the potential for obscure bugs which could lose me data.

2bluesc 9y ago

The results don't look very complete seeing as how attic was abandoned over two years ago. The master attic branch[0] has 600 commits. The fork of attic, borg, has over 4000 commits [1] suggesting a significant amount of work has been done to improve it. It seems odd for the author to compare it to something abandoned (and thankfully reborn as borg) and ignore what has happened in two years. Would love to see similar tests run against borg. [0] https://github.com/jborg/attic https://github.com/jborg/attic [1] https://github.com/borgbackup/borg https://github.com/borgbackup/borg

zokier 9y ago
The version row for attic says "BorgBackup 1.1.0b6"
acrosync 9y ago
Author here. The experiments actually ran BorgBackup 1.1.0b6 as you can see from the Setup section. We liked to call it Attic out of the respect to the original Attic author.
- tombrossman 9y ago
  You realize the name Borg comes from the Attic author, Jonas Borgström, right? No one else calls it Attic as the two are different projects.
  - acrosync 9y ago
    I noticed that, but didn't know if Borg has another meaning. I can understand why they forked the project, but in my opinion a name that makes the origin more obvious would have been better.
    - dom0 9y ago
      "Borg" was chosen, because it emphasizes collaborative development — and because someone is a Star Trek fan ;)

luxpir 9y ago

Could obnam be added? I use it and with a few tweaks it's very respectable in terms of performance. Also has some good integrity checks.

ams6110 9y ago

> duplicity has a serious flaw in its incremental model -- the user has to decide whether to perform a full backup or an incremental backup on each run. That is because while an incremental backup saves a lot of storage space, it is also dependent on previous backups due to the design of duplicity, making it impossible to delete any single backup on a long chain of dependent backups. So there is always a dilemma of how often to perform a full backup for duplicity users. Yes and no. Duplicity has the "--full-if-older-than" option so you can do incrementals normally, but if your previous full is older than whatever interval you define, it will do a full backup, without changing the command line. So that can be e.g. in a cron job.

willvarfar 9y ago
Classic source control had this problem. The clever trick is to reencode the previous most-recent backup as a delta from the current state, and do a full-backup of the current state, rather than encoding each new backup as a delta from the previous state (which becomes slower and slower to compute, the more previous states you have). Problem solved :)
- tatersolid 9y ago
  That's really expensive when your previous backup is on cloud storage
  - willvarfar 9y ago
    To compute a delta, you need the previous version. If this previous version is computed from a single file - the previous snapshot - then that's actually less data and effort than if its computed by taking an old snapshot and replaying all the deltas upto the current time.

beagle3 9y ago

i think bup does support concurrent access; the concurrent deduplication granularity is a backup set, so if two identical computers are backed up for the first time at exactly the same time you will not get deduplication - but that's inherently a dining philosopher kind of problem Also, recent bup versions allow delete. No encryption IiRC, but you can examine it with git tools which is a feature on its own.

AdmiralAsshat 9y ago

I would've been curious to see how BackinTime stacks up. Duplicity/deja-dup (a GNOME frontend for duplicity) is pre-installed on most GNOME-based DE's, which makes it convenient for end-users, but I found the limitation of only being able to configure a single backup destination to be too limiting. By contrast, BiT supported multiple destinations and profiles, meaning I could have one local, one off-site, one "Personal Data" backup, one "System" backup to fall back if an OS update fails, etc. Its configuration options were much more attractive.

e12e 9y ago

It's a little odd to not benchmark backup over the network - a backup taken to the same physical disk as the source of the data isn't very useful - for that use-case taking a filesystem snapshot[s] would probably be faster and more useful. Perhaps in combination with a checksumming tool, like [c], or with a filesystem like ZFS. Also, it can be difficult in a lot of environments to sustain more than 100mbps write to a remote, off-site system - halving the stored data can be a much bigger win then. All that said, it's interesting to see that a) duplicity seems slow, and b) very consistent in terms of speed. I wonder if there's some low-hanging fruit for optimization there. Personally I've had some luck using backupninja[b] in combination with duplicity. It's one of the few Free alternatives that allow the backup-system to encrypt "one-way" - so that compromising the backup-system doesn't immediately give read access to encrypted backups. It's a bit complicated to set it up for separate encrypt-to and signing keys though :/ [s] Today I would probably recommend ZFS - but I've always wanted to give NILFS2 a real test, especially on solid-state disks: http://nilfs.osdn.jp/en/ http://nilfs.osdn.jp/en/ [c] https://github.com/Tripwire/tripwire-open-source https://github.com/Tripwire/tripwire-open-source http://aide.sourceforge.net/ http://aide.sourceforge.net/ https://github.com/integrit/integrit https://github.com/integrit/integrit (Speaking of projects that might be fun/useful to redo in a safe language like rust or go - it would appear this would be a prime example, btw. On the whole moving integrity to the fs, as with zfs might be the better option, though). [b] https://0xacab.org/riseuplabs/backupninja https://0xacab.org/riseuplabs/backupninja

dom0 9y ago
> All that said, it's interesting to see that a) duplicity seems slow, and b) very consistent in terms of speed. I wonder if there's some low-hanging fruit for optimization there. Duplicity is classic delta-backup. It always reads all files and calculates a delta to a different version of the file, hence the fairly consistent performance. Performance of deduplicating archivers is more difficult to predict.

amq 9y ago

If you are using duplicity, use this moment to check if it works. There is a serious bug which seems to affect systems with a lot of data: https://bugs.launchpad.net/duplicity/+bug/896728 https://bugs.launchpad.net/duplicity/+bug/896728

dpc_pw 9y ago

I wonder how rdedup would compare.

dom0 9y ago

My two (possibly biased, much like the author's) cents. - No network-based tests; e.g. a typical fast internet connection (say 100/40 or 50/20 MBit/s) with a few dozen ms latency to some server or cloud service. This is of course difficult because these tend to be bad on reproducibility. For a network-based test, not only time is interesting, but total RX/TX as well. - I'm really surprised at restic's performance. It uses far more CPU than Borg in almost all tests... and Borg is already notoriously inefficient in it's CPU usage when looking at object throughput (restic: "fast, efficient"?). I don't mean to bash, I'm just surprised. - restic's deduplication performance might hint at Rabin Fingerprints being worse than Buzhash, but there might be other issue(s) leading to this result. - Besides CPU time, memory (peak) usage would be interesting. > For instance, file hashes enable users to quickly identify which files in existing backups are changed. They also allow third-party tools to compare files on disks to those in the backups. To be fair, Borg can calculate a variety of file hashes (MD5, SHA1, SHA2, ...) on the fly with "borg list". There are "borg diff" (to compare two archives) and "borg mount -o versions" as well, though the latter is generally impractical for looking at a large number of archives. > Again, by not computing the file hash helped improve the performance, but at the risk of possible undetected data corruption. I can't deduce how the last part follows (", but..."). Care to explain?