20 ms·

Deduplicating Archiver with Compression and Encryption

134 points by jiehong 5y ago

jlelse 5y ago
What about Borg vs Restic? Which is better?
- formerly_proven 5y ago
  Restic doesn't support compression and probably won't. https://github.com/restic/restic/issues/21 https://github.com/restic/restic/issues/21 Borg has a "lackluster" encryption scheme. https://borgbackup.readthedocs.io/en/stable/internals/security.html https://borgbackup.readthedocs.io/en/stable/internals/securi... Restic can't pull backups. https://github.com/restic/restic/issues/299 https://github.com/restic/restic/issues/299 Borg can't either, but it has "borg import-tar", which might be good enough.
  - Freak_NL 5y ago
    Could you explain why you find their encryption scheme 'lackluster'? I'm not seeing that conclusion in the linked document.
    - formerly_proven 5y ago
      Borg uses static AES keys and requires a way to store nonces because of this. In a multi-client scenario there is no way to do that without trusting the server to some extent, so the encryption is vulnerable to nonce reuse. > When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more).
      - ThomasWaldmann 5y ago
        Yeah, that's an issue which can be avoided by using 1 repo per client. There are some ideas on the issue tracker for fixing this long term (like random nonces, session keys, ...), but that stuff will have to wait until after borg 1.2 (which soon goes into release candidate phase).
  - dmos62 5y ago
    > Restic can't pull backups There's ways to pull backups. You'd use ssh forwarding and/or the REST backend for append-only backups. See this comment: https://github.com/restic/restic/issues/299#issuecomment-456310904 https://github.com/restic/restic/issues/299#issuecomment-456...
- aborsy 5y ago
  A major difference is that, Borg needs a server side code running as well, unlike restic. Restic can take up a lot of RAM, unlike Borg. Borg offers the option of no encryption. This is useful when backing up to a local drive that is already encrypted, for example with LUKS. But, they are generally similar.
  - beermonster 5y ago
    Borg doesn’t require server side support. You can mount locally using, say, SSHFS. However having server side support reduces backup times.
    - rsync 5y ago
      I think you are both not quite right ... Which is to say, you don't need the borg executable server-side if you are happy to run it over SFTP transport which limits some functionality. Yes, of course you could talk over an sshfs mount point but I can't think of why you would do that over the more basic SFTP transport. The accepted, and full-featured, way to run borg is with the executable on the server side and I will point out that the borg project distributes a "frozen" version of the tool that allows you to run it without having python in your environment. I believe they use py2exe to pack it up. This is important for us[1] as we have no interpreters of any kind in our environment (no shell, no python, no perl) so we can only run binary executables ... [1] rsync.net
      - beermonster 5y ago
        In the FAQ [1] SSHFS is mentioned a few times and here is the citation where I got this idea from when I read it way back when: “When Borg is writing to a repo on a locally mounted remote file system, e.g. SSHFS, the Borg client only can do file system operations and has no agent running on the remote side, so every operation needs to go over the network, which is slower.” SSHFS would be tool agnostic/transparent though of course would result in operations going over the network as it’s not a local repo, but pseudo-local. I’m curious, how do you use plain SFTP transport with borg using rsync.net? [1] https://borgbackup.readthedocs.io/en/stable/faq.html https://borgbackup.readthedocs.io/en/stable/faq.html
- beermonster 5y ago
  Restic runs natively on Windows too which is useful in heterogenous environments Filippo Valsorda thinks[1] the encryption implementation looks sane. [1] https://blog.filippo.io/restic-cryptography/ https://blog.filippo.io/restic-cryptography/
- hashhar 5y ago
  Restic doesn't implement compression and most probably never will (since compression when combined with encryption has been known to make the encryption "weaker"). Borg's "encryption" doesn't instill confidence in me. Restic ships as a static binary so you can shove the version you used to create the backup and will be able to restore it forever (assuming we still run x64 machines). Borg requires a server side component so cannot natively backup to cloud object stores. Restic can. Borg OTOH is very lean and doesn't take a lot of RAM compared to Restic. You cannot go wrong with either. I tried both and found Restic much easier to run and manage and the memory usage wasn't an issue for me (~4TB backups).
  - formerly_proven 5y ago
    I don't think the reason Restic won't support compression is because of some security concerns (Restic encrypts blobs separately, and you'd compress things blob-wise as well), but because there simply wasn't any space designed into their file formats to signal compression, so it's essentially impossible to implement it without breaking compatibility. Borg's story is actually similar but the opposite. Attic only used to support zlib, other methods were added later. This was possible because the zlib header uses only a few values for the first two bytes, so there is enough room to indicate various compression formats.
  - jlelse 5y ago
    I use restic as well, was just interested to see what's different about Borg. But I think restic was the right choice, already saved me once from data loss.
    - beermonster 5y ago
      They’re both a fine choice really. And being open source there’s no harm (other than time) in trying both and seeing which suits your use case best
  - thde 5y ago
    We found that restic (at least in its newest version) uses ~ the same amount of RAM or even less that Borg on some systems. But no big differences there.
- ale42 5y ago
  I wont comment in terms of features, but in my experience Borg is slower than Restic. This might be worth considering for low-end servers, and for big amount of data. Restic used to have an extremely slow prune (purge unused old data) feature, but this has improved dramatically in one of the latest releases.
jhoechtl 5y ago
And the contender is restic https://restic.net/ https://restic.net/
- beagle3 5y ago
  It is awesome. I like it better. It only deduplicates though. Does not compress.
- ThomasWaldmann 5y ago
  borg maintainer here: Yeah, restic is a great backup tool also. OTOH it is a pity the we could not join forces (Python vs. Go and also some other differences), but otoh it means more choice for everybody wanting a nice deduplicating backup tool.
fragileone 5y ago
Very content with Borg for my backups, using it via the Vorta GUI client [1]. Zero issues for me unlike Duplicati and Duplicacy. [1] https://github.com/borgbase/vorta https://github.com/borgbase/vorta
- Yeroc 5y ago
  I'm curious what your issues were with Duplicacy. I looked at Borg, Duplicati and Duplicacy among others when I was evaluating a backup solution and eventually settled on Duplicacy and have been pretty happy with it. The only issue I've had is that it's a bit memory hungry during backups.
  - fragileone 5y ago
    1) The web app GUI is bad - can't reorder saved options/backups/etc, each of these options use up too much screen space and there's no compact mode or minimising the options, it could be more user-friendly especially with being able to add arguments with the GUI rather than text. 2) It's proprietary with source-available but not open-source. This makes me hesitate to rely on it in the long-term as I don't know if it'll remain supported, especially considering that: 3) Development speed was very slow when I dropped Duplicacy about 6 months ago, and seeing Borg have many contributors makes me think it's more likely to stick around in the long-term. 4) GUI doesn't give much insight into the status of the backup other than progress %. 5) Restore operations are easier and quicker with Borg/Vorta because you can mount the backup via FUSE.
m3nu 5y ago
There is also Borgmatic, which makes it easy to manage backup settings, do monitoring, directly dump databases and prune old archives with a single YAML file. https://torsion.org/borgmatic/ https://torsion.org/borgmatic/ We also maintain an Ansible role to set it up quickly on new servers. https://github.com/borgbase/ansible-role-borgbackup https://github.com/borgbase/ansible-role-borgbackup And last, if you need a place to put your backups, try my https://borgbase.com https://borgbase.com offering. It's purpose-built for Borg and offers handy features, like separating each repository and monitoring for stale backups.
- witten 5y ago
  Thanks for the shout out, Manu! I'll also plug an all-in one Borg + borgmatic Docker image (https://hub.docker.com/r/b3vis/borgmatic/ https://hub.docker.com/r/b3vis/borgmatic/) that I've used to backup to both BorgBase and rsync.net.
Arkanosis 5y ago
After years of looking at it, I've been using borg for almost a year now, and I've already replaced rsnapshot with it pretty much everywhere. I'm very happy with the deduplication feature (that's my main motivation for switching), but the performance and ease of use are very appreciable too. Actually, performance might be as much as a game changer for me as deduplication (though a less expected one), as I find myself doing and automating backups way more often than before. I also enjoy the ability to do append-only backups which mitigate the risks of losing already save data because of bug / malware. I've been expecting more and more from it, and at this point, there are only two things I wish would be better supported: - deduplication across several machines (you can already backup several machines to the same repository, but it's not efficient); - builtin redundancy to deal with bitrot. I'm still using rsnapshot for secondary backups, as I can't afford a bug in backup software, but I'm considering switching to restic for that, as it provides deduplication as well and doesn't share code with borg.
- hrez 5y ago
  check out https://github.com/kopia/kopia https://github.com/kopia/kopia https://kopia.io/docs/features/ https://kopia.io/docs/features/
  - bori5 5y ago
    Haven’t tried any backup solution for my desktop other than some external SSDs I copied to a few times, yeah I know, how does borg compare to Kopia for home user who has maybe a few TB ? Or in that regards to timemachine which I’ve never used.
  - khimaros 5y ago
    i'm a happy borg backup user, but this looks really promising. thanks for sharing. have the developers made any commitments to storage format stability?
    - gloriousternary 5y ago
      https://kopia.io/docs/advanced/compatibility/ https://kopia.io/docs/advanced/compatibility/
rom1v 5y ago
I use borg since recently, it's awesome. I also like how it is possible to navigate into archives by just mounting them in the filesystem: https://borgbackup.readthedocs.io/en/stable/usage/mount.html https://borgbackup.readthedocs.io/en/stable/usage/mount.html
rom1v 5y ago
If you want to (also) backup remotely (the server don't have access to encrypted data), I would recommend some borg-specific hosts: - Rsync.net: https://www.rsync.net/ https://www.rsync.net/ - BorgBase: https://www.borgbase.com/ https://www.borgbase.com/ - Lima-Labs: https://storage.lima-labs.com/ https://storage.lima-labs.com/ - others? (I'm not affiliated in any way, I'm just a client of one of them.)
- martijnvds 5y ago
  Hetzner's "storage box" product also supports Borg backups: https://www.hetzner.com/storage/storage-box https://www.hetzner.com/storage/storage-box
  - sildur 5y ago
    Thanks! I had no idea, this is going to simplify my backups a lot.
- wasmitnetzen 5y ago
  Mhm, BorgBase looks a bit fishy. It is based in Malta, but why? For "tax efficiency"? They share the office address with [1] and the CEO of the parent company PeakFord[2] seems to be a CEO-for-hire[3]. Their GDPR page[4] mentions that they are not based in the EU and want to use the German privacy regulator. Malta is very much part of the EU. [1]: https://opes.com.mt/corporate-services/ https://opes.com.mt/corporate-services/ [2]: https://www.peakford.com/about/ https://www.peakford.com/about/ [3]: https://offshoreleaks.icij.org/nodes/56060344 https://offshoreleaks.icij.org/nodes/56060344 [4]: https://www.borgbase.com/gdpr https://www.borgbase.com/gdpr
  - m3nu 5y ago
    BorgBase founder here. All correct what you found. I started the company from the US via YC startup school in 2018, but later decided to place myself in Europe again. Hence moving the incorporation to the EU made sense. My director is handling the commercial aspect, like VAT, taxes and banking. Myself, I maintain our open source projects like Vorta, our Ansible role and macOS packaging for Borg. Hope this helps to clear your reservations. I should update our GDPR page with regard to the regulator. That's outdated by now.
rvanlaar 5y ago
I've been using borg now for a couple of years. It has been great. I previously used duplicity. It would do a full backup once every week or month, resulting in GBs of data being send over the line. Borg chunks the data and uses deduplication. It result is just one huge backup at the beginning and incremental changes afterwards. The abstractions used are also much friendlier. I never took the time to retrieve specific point in time backups from duplicity. With Borg one can mount the whole backup repository as a fuse mount, which each different backup being a distinct directory. rsync.net offers cheap online storage for borg backups. One caveat, borg is not yet multiprocess.
pt_PT_guy 5y ago
For the lazy like me, there's this: https://vorta.borgbase.com/ https://vorta.borgbase.com/
Hugie 5y ago
Is it comparable with exdupe? http://www.quicklz.com/exdupe/ http://www.quicklz.com/exdupe/
- ThomasWaldmann 5y ago
  I don't know exdupe, but from a quick glance at their homepage it looks like a full + differential style backup. With borg, every backup is logically a full backup (it just is faster due to deduplication). Much easier and less time consuming to deal with.
MikusR 5y ago
If you need Windows or GUI support there is Kopia https://news.ycombinator.com/item?id=27471945 https://news.ycombinator.com/item?id=27471945
sandreas 5y ago
And yet again: A list of alternatives. attic (python) - https://github.com/jborg/attic https://github.com/jborg/attic borg (c) - https://github.com/borgbackup/borg https://github.com/borgbackup/borg bupstash (rust) - https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash duplicacy (go) - https://github.com/gilbertchen/duplicacy https://github.com/gilbertchen/duplicacy duplicati (c#) - https://github.com/duplicati/duplicati https://github.com/duplicati/duplicati duplicity (python) - https://github.com/henrysher/duplicity https://github.com/henrysher/duplicity kopia (go) - https://github.com/kopia/kopia https://github.com/kopia/kopia nfreezer (python) - https://github.com/josephernest/nfreezer https://github.com/josephernest/nfreezer rdedup (rust) - https://github.com/dpc/rdedup https://github.com/dpc/rdedup restic (go) - https://github.com/restic/restic https://github.com/restic/restic rclone (go) - https://github.com/rclone/rclone https://github.com/rclone/rclone rsnapshot (perl) - https://github.com/rsnapshot/rsnapshot https://github.com/rsnapshot/rsnapshot snebu (c) - https://github.com/derekp7/snebu https://github.com/derekp7/snebu tarsnap (c) - https://github.com/Tarsnap/tarsnap https://github.com/Tarsnap/tarsnap I think there are many more out there (https://github.com/restic/others https://github.com/restic/others) - I personally use restic while technology wise (speed, only restore needs password) i would prefer rdedup which is an impressive piece of software but unfortunately without file iterator... :-)
- abecode 5y ago
  would casync [1] fit in this list? [1] https://github.com/systemd/casync/ https://github.com/systemd/casync/
- ThomasWaldmann 5y ago
  borg maintainer here: attic should not be on the list, it is unmaintained since 2015. borg came into life as a fork of attic for related reasons, so it should be borg (Python+Cython+C).
indiv0 5y ago
I guess I'll ask here since I'm in the process of updating my backup system. I have a variety of machines I'd like to backup (desktops/servers/laptops/phones) running a variety of OSes. I have a NAS ZFS machine that I'd like to host a copy of all my data from each machine. From the NAS, I'd like to backup to a cloud host (e.g. rsync.net). What I'm unsure about is where to introduce Borg in this scheme. I see a few permutations: 1) Borg from each machine to the NAS, then rsync to rsync.net 2) Rsync/ZFS send/etc. from each machine to the NAS, then Borg to a different location on the NAS, then rsync the Borg repo to rsync.net 3) Rsync/ZFS send/etc. from each machine to the NAS, then Borg directly to rsync.net I'm leaning towards #3 personally. Thoughts?
- pnutjam 5y ago
  I'd say this depend on where you want to introduce the encryption and de-duplication. I'd lean towards 1, so each machine has a standalone encrypted backup, but 3 would provide easier access without the borg client for local backups, and better de-duplication if files are shared across machines.
- fragileone 5y ago
  I'd prefer 1, as it allows each machine to easily and quickly restore itself directly from a local backup. This way the ZFS and rsync.net backup components are only used incase of catastrophic failure.
- infogulch 5y ago
  If you have ZFS clients and server, what benefit does inserting borg provide on top of a continuous ZFS send from client -> server and server -> rsync.net? ZFS send can already do encryption, and ZFS supports compression. And snapshots automatically "dedups" files that haven't changed between snapshot points, which I'd guess saves more space than deduping literal duplicate files which are not very common ime. I guess I just don't see what value borg provides if you're already using ZFS throughout.
  - indiv0 5y ago
    I guess my concern is that I'd like the data to be encrypted at rsync.net, which means I'd need to use raw ZFS send. Would it be possible to mount/browse those raw datasets remotely to verify my backups? Or would I need to recv it all back to be able to test them?
l1am0 5y ago
Love borg with the desktop client vorta. Gives my time-machine like time travel :D Even wrote a blog post about it: https://simon-frey.com/blog/borgvorta-is-finally-a-usable-backup-solution-for-linux-desktop/ https://simon-frey.com/blog/borgvorta-is-finally-a-usable-ba...
spaetzleesser 5y ago
Since I moved my personal laptop from Mac to Linux I have been using borg. It works really well so far. One thing that’s confusing me is the prune strategy. It seems prune removes the whole backup set at once. Let’s say I have daily backups and weekly backups. At some point I am pruning the daily backups. It seems this means that if the weekly backup ran on Sunday but a file was created on Monday and deleted on Friday, the file would be completely deleted during prune and there would be no trace of it. I would much prefer if the pruning was done on per file basis and not by pruning whole backup sets.
- ThomasWaldmann 5y ago
  Maybe have a look at borgbackup docs to use the same terminology, otherwise it might get confusing. When you create a backup ARCHIVE in a borg REPOSITORY, the archive contains all input files you gave to borg. Each archive has a name and usually the name is something like machinename-setname-date-time. You can create such backups rather often (that is cheap, due to deduplication), but you do not want to keep tons of archives long term (some borg commands take O(archive count) time). Thus, one runs borg prune to "thin out" archives and only keep some following some prune policy, like keeping 60 daily, 12 monthly, 10 yearly or so. It is important to use --prefix if one creates multiple different archive sequences in the same repo, e.g. if you create pc-home-date-time as well as pc-system-date-time - then you need to run prune with --prefix=pc-home- and another prune with --prefix=pc-system- . Usually, archives are not modified after their creation (the only way to do that is to carefully use borg recreate command).
- axiolite 5y ago
  > I would much prefer if the pruning was done on per file basis and not by pruning whole backup sets. Then just don't prune! Unchanged files take no space. There's no reason you can't keep many months of daily backups, and set your prune to only do: --keep-within=365d For those who aren't familiar with Borg, there is no such thing as a weekly backup, daily backup, monthly backup, etc. Every backup is a quick full backup. It's only the prune (which deletes old backups) option that gives you the choice of bracketing your backups into monthly/weekly/daily sets, so it'll save the last backup of each month for the last X months, the last backup of each week for X weeks, etc.

134 points by jiehong 5y ago

jlelse 5y ago

What about Borg vs Restic? Which is better?

formerly_proven 5y ago
Restic doesn't support compression and probably won't. https://github.com/restic/restic/issues/21 https://github.com/restic/restic/issues/21 Borg has a "lackluster" encryption scheme. https://borgbackup.readthedocs.io/en/stable/internals/security.html https://borgbackup.readthedocs.io/en/stable/internals/securi... Restic can't pull backups. https://github.com/restic/restic/issues/299 https://github.com/restic/restic/issues/299 Borg can't either, but it has "borg import-tar", which might be good enough.
- Freak_NL 5y ago
  Could you explain why you find their encryption scheme 'lackluster'? I'm not seeing that conclusion in the linked document.
  - formerly_proven 5y ago
    Borg uses static AES keys and requires a way to store nonces because of this. In a multi-client scenario there is no way to do that without trusting the server to some extent, so the encryption is vulnerable to nonce reuse. > When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more).
    - ThomasWaldmann 5y ago
      Yeah, that's an issue which can be avoided by using 1 repo per client. There are some ideas on the issue tracker for fixing this long term (like random nonces, session keys, ...), but that stuff will have to wait until after borg 1.2 (which soon goes into release candidate phase).
- dmos62 5y ago
  > Restic can't pull backups There's ways to pull backups. You'd use ssh forwarding and/or the REST backend for append-only backups. See this comment: https://github.com/restic/restic/issues/299#issuecomment-456310904 https://github.com/restic/restic/issues/299#issuecomment-456...
aborsy 5y ago
A major difference is that, Borg needs a server side code running as well, unlike restic. Restic can take up a lot of RAM, unlike Borg. Borg offers the option of no encryption. This is useful when backing up to a local drive that is already encrypted, for example with LUKS. But, they are generally similar.
- beermonster 5y ago
  Borg doesn’t require server side support. You can mount locally using, say, SSHFS. However having server side support reduces backup times.
  - rsync 5y ago
    I think you are both not quite right ... Which is to say, you don't need the borg executable server-side if you are happy to run it over SFTP transport which limits some functionality. Yes, of course you could talk over an sshfs mount point but I can't think of why you would do that over the more basic SFTP transport. The accepted, and full-featured, way to run borg is with the executable on the server side and I will point out that the borg project distributes a "frozen" version of the tool that allows you to run it without having python in your environment. I believe they use py2exe to pack it up. This is important for us[1] as we have no interpreters of any kind in our environment (no shell, no python, no perl) so we can only run binary executables ... [1] rsync.net
    - beermonster 5y ago
      In the FAQ [1] SSHFS is mentioned a few times and here is the citation where I got this idea from when I read it way back when: “When Borg is writing to a repo on a locally mounted remote file system, e.g. SSHFS, the Borg client only can do file system operations and has no agent running on the remote side, so every operation needs to go over the network, which is slower.” SSHFS would be tool agnostic/transparent though of course would result in operations going over the network as it’s not a local repo, but pseudo-local. I’m curious, how do you use plain SFTP transport with borg using rsync.net? [1] https://borgbackup.readthedocs.io/en/stable/faq.html https://borgbackup.readthedocs.io/en/stable/faq.html
beermonster 5y ago
Restic runs natively on Windows too which is useful in heterogenous environments Filippo Valsorda thinks[1] the encryption implementation looks sane. [1] https://blog.filippo.io/restic-cryptography/ https://blog.filippo.io/restic-cryptography/
hashhar 5y ago
Restic doesn't implement compression and most probably never will (since compression when combined with encryption has been known to make the encryption "weaker"). Borg's "encryption" doesn't instill confidence in me. Restic ships as a static binary so you can shove the version you used to create the backup and will be able to restore it forever (assuming we still run x64 machines). Borg requires a server side component so cannot natively backup to cloud object stores. Restic can. Borg OTOH is very lean and doesn't take a lot of RAM compared to Restic. You cannot go wrong with either. I tried both and found Restic much easier to run and manage and the memory usage wasn't an issue for me (~4TB backups).
- formerly_proven 5y ago
  I don't think the reason Restic won't support compression is because of some security concerns (Restic encrypts blobs separately, and you'd compress things blob-wise as well), but because there simply wasn't any space designed into their file formats to signal compression, so it's essentially impossible to implement it without breaking compatibility. Borg's story is actually similar but the opposite. Attic only used to support zlib, other methods were added later. This was possible because the zlib header uses only a few values for the first two bytes, so there is enough room to indicate various compression formats.
- jlelse 5y ago
  I use restic as well, was just interested to see what's different about Borg. But I think restic was the right choice, already saved me once from data loss.
  - beermonster 5y ago
    They’re both a fine choice really. And being open source there’s no harm (other than time) in trying both and seeing which suits your use case best
- thde 5y ago
  We found that restic (at least in its newest version) uses ~ the same amount of RAM or even less that Borg on some systems. But no big differences there.
ale42 5y ago
I wont comment in terms of features, but in my experience Borg is slower than Restic. This might be worth considering for low-end servers, and for big amount of data. Restic used to have an extremely slow prune (purge unused old data) feature, but this has improved dramatically in one of the latest releases.

jhoechtl 5y ago

And the contender is restic https://restic.net/ https://restic.net/

beagle3 5y ago
It is awesome. I like it better. It only deduplicates though. Does not compress.
ThomasWaldmann 5y ago
borg maintainer here: Yeah, restic is a great backup tool also. OTOH it is a pity the we could not join forces (Python vs. Go and also some other differences), but otoh it means more choice for everybody wanting a nice deduplicating backup tool.

fragileone 5y ago

Very content with Borg for my backups, using it via the Vorta GUI client [1]. Zero issues for me unlike Duplicati and Duplicacy. [1] https://github.com/borgbase/vorta https://github.com/borgbase/vorta

Yeroc 5y ago
I'm curious what your issues were with Duplicacy. I looked at Borg, Duplicati and Duplicacy among others when I was evaluating a backup solution and eventually settled on Duplicacy and have been pretty happy with it. The only issue I've had is that it's a bit memory hungry during backups.
- fragileone 5y ago
  1) The web app GUI is bad - can't reorder saved options/backups/etc, each of these options use up too much screen space and there's no compact mode or minimising the options, it could be more user-friendly especially with being able to add arguments with the GUI rather than text. 2) It's proprietary with source-available but not open-source. This makes me hesitate to rely on it in the long-term as I don't know if it'll remain supported, especially considering that: 3) Development speed was very slow when I dropped Duplicacy about 6 months ago, and seeing Borg have many contributors makes me think it's more likely to stick around in the long-term. 4) GUI doesn't give much insight into the status of the backup other than progress %. 5) Restore operations are easier and quicker with Borg/Vorta because you can mount the backup via FUSE.

m3nu 5y ago

There is also Borgmatic, which makes it easy to manage backup settings, do monitoring, directly dump databases and prune old archives with a single YAML file. https://torsion.org/borgmatic/ https://torsion.org/borgmatic/ We also maintain an Ansible role to set it up quickly on new servers. https://github.com/borgbase/ansible-role-borgbackup https://github.com/borgbase/ansible-role-borgbackup And last, if you need a place to put your backups, try my https://borgbase.com https://borgbase.com offering. It's purpose-built for Borg and offers handy features, like separating each repository and monitoring for stale backups.

witten 5y ago
Thanks for the shout out, Manu! I'll also plug an all-in one Borg + borgmatic Docker image (https://hub.docker.com/r/b3vis/borgmatic/ https://hub.docker.com/r/b3vis/borgmatic/) that I've used to backup to both BorgBase and rsync.net.

Arkanosis 5y ago

After years of looking at it, I've been using borg for almost a year now, and I've already replaced rsnapshot with it pretty much everywhere. I'm very happy with the deduplication feature (that's my main motivation for switching), but the performance and ease of use are very appreciable too. Actually, performance might be as much as a game changer for me as deduplication (though a less expected one), as I find myself doing and automating backups way more often than before. I also enjoy the ability to do append-only backups which mitigate the risks of losing already save data because of bug / malware. I've been expecting more and more from it, and at this point, there are only two things I wish would be better supported: - deduplication across several machines (you can already backup several machines to the same repository, but it's not efficient); - builtin redundancy to deal with bitrot. I'm still using rsnapshot for secondary backups, as I can't afford a bug in backup software, but I'm considering switching to restic for that, as it provides deduplication as well and doesn't share code with borg.

hrez 5y ago
check out https://github.com/kopia/kopia https://github.com/kopia/kopia https://kopia.io/docs/features/ https://kopia.io/docs/features/
- bori5 5y ago
  Haven’t tried any backup solution for my desktop other than some external SSDs I copied to a few times, yeah I know, how does borg compare to Kopia for home user who has maybe a few TB ? Or in that regards to timemachine which I’ve never used.
- khimaros 5y ago
  i'm a happy borg backup user, but this looks really promising. thanks for sharing. have the developers made any commitments to storage format stability?
  - gloriousternary 5y ago
    https://kopia.io/docs/advanced/compatibility/ https://kopia.io/docs/advanced/compatibility/

rom1v 5y ago

I use borg since recently, it's awesome. I also like how it is possible to navigate into archives by just mounting them in the filesystem: https://borgbackup.readthedocs.io/en/stable/usage/mount.html https://borgbackup.readthedocs.io/en/stable/usage/mount.html

rom1v 5y ago

If you want to (also) backup remotely (the server don't have access to encrypted data), I would recommend some borg-specific hosts: - Rsync.net: https://www.rsync.net/ https://www.rsync.net/ - BorgBase: https://www.borgbase.com/ https://www.borgbase.com/ - Lima-Labs: https://storage.lima-labs.com/ https://storage.lima-labs.com/ - others? (I'm not affiliated in any way, I'm just a client of one of them.)

martijnvds 5y ago
Hetzner's "storage box" product also supports Borg backups: https://www.hetzner.com/storage/storage-box https://www.hetzner.com/storage/storage-box
- sildur 5y ago
  Thanks! I had no idea, this is going to simplify my backups a lot.
wasmitnetzen 5y ago
Mhm, BorgBase looks a bit fishy. It is based in Malta, but why? For "tax efficiency"? They share the office address with [1] and the CEO of the parent company PeakFord[2] seems to be a CEO-for-hire[3]. Their GDPR page[4] mentions that they are not based in the EU and want to use the German privacy regulator. Malta is very much part of the EU. [1]: https://opes.com.mt/corporate-services/ https://opes.com.mt/corporate-services/ [2]: https://www.peakford.com/about/ https://www.peakford.com/about/ [3]: https://offshoreleaks.icij.org/nodes/56060344 https://offshoreleaks.icij.org/nodes/56060344 [4]: https://www.borgbase.com/gdpr https://www.borgbase.com/gdpr
- m3nu 5y ago
  BorgBase founder here. All correct what you found. I started the company from the US via YC startup school in 2018, but later decided to place myself in Europe again. Hence moving the incorporation to the EU made sense. My director is handling the commercial aspect, like VAT, taxes and banking. Myself, I maintain our open source projects like Vorta, our Ansible role and macOS packaging for Borg. Hope this helps to clear your reservations. I should update our GDPR page with regard to the regulator. That's outdated by now.

rvanlaar 5y ago

I've been using borg now for a couple of years. It has been great. I previously used duplicity. It would do a full backup once every week or month, resulting in GBs of data being send over the line. Borg chunks the data and uses deduplication. It result is just one huge backup at the beginning and incremental changes afterwards. The abstractions used are also much friendlier. I never took the time to retrieve specific point in time backups from duplicity. With Borg one can mount the whole backup repository as a fuse mount, which each different backup being a distinct directory. rsync.net offers cheap online storage for borg backups. One caveat, borg is not yet multiprocess.

pt_PT_guy 5y ago

For the lazy like me, there's this: https://vorta.borgbase.com/ https://vorta.borgbase.com/

Hugie 5y ago

Is it comparable with exdupe? http://www.quicklz.com/exdupe/ http://www.quicklz.com/exdupe/

ThomasWaldmann 5y ago
I don't know exdupe, but from a quick glance at their homepage it looks like a full + differential style backup. With borg, every backup is logically a full backup (it just is faster due to deduplication). Much easier and less time consuming to deal with.

MikusR 5y ago

If you need Windows or GUI support there is Kopia https://news.ycombinator.com/item?id=27471945 https://news.ycombinator.com/item?id=27471945

sandreas 5y ago

And yet again: A list of alternatives. attic (python) - https://github.com/jborg/attic https://github.com/jborg/attic borg (c) - https://github.com/borgbackup/borg https://github.com/borgbackup/borg bupstash (rust) - https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash duplicacy (go) - https://github.com/gilbertchen/duplicacy https://github.com/gilbertchen/duplicacy duplicati (c#) - https://github.com/duplicati/duplicati https://github.com/duplicati/duplicati duplicity (python) - https://github.com/henrysher/duplicity https://github.com/henrysher/duplicity kopia (go) - https://github.com/kopia/kopia https://github.com/kopia/kopia nfreezer (python) - https://github.com/josephernest/nfreezer https://github.com/josephernest/nfreezer rdedup (rust) - https://github.com/dpc/rdedup https://github.com/dpc/rdedup restic (go) - https://github.com/restic/restic https://github.com/restic/restic rclone (go) - https://github.com/rclone/rclone https://github.com/rclone/rclone rsnapshot (perl) - https://github.com/rsnapshot/rsnapshot https://github.com/rsnapshot/rsnapshot snebu (c) - https://github.com/derekp7/snebu https://github.com/derekp7/snebu tarsnap (c) - https://github.com/Tarsnap/tarsnap https://github.com/Tarsnap/tarsnap I think there are many more out there (https://github.com/restic/others https://github.com/restic/others) - I personally use restic while technology wise (speed, only restore needs password) i would prefer rdedup which is an impressive piece of software but unfortunately without file iterator... :-)

abecode 5y ago
would casync [1] fit in this list? [1] https://github.com/systemd/casync/ https://github.com/systemd/casync/
ThomasWaldmann 5y ago
borg maintainer here: attic should not be on the list, it is unmaintained since 2015. borg came into life as a fork of attic for related reasons, so it should be borg (Python+Cython+C).

indiv0 5y ago

I guess I'll ask here since I'm in the process of updating my backup system. I have a variety of machines I'd like to backup (desktops/servers/laptops/phones) running a variety of OSes. I have a NAS ZFS machine that I'd like to host a copy of all my data from each machine. From the NAS, I'd like to backup to a cloud host (e.g. rsync.net). What I'm unsure about is where to introduce Borg in this scheme. I see a few permutations: 1) Borg from each machine to the NAS, then rsync to rsync.net 2) Rsync/ZFS send/etc. from each machine to the NAS, then Borg to a different location on the NAS, then rsync the Borg repo to rsync.net 3) Rsync/ZFS send/etc. from each machine to the NAS, then Borg directly to rsync.net I'm leaning towards #3 personally. Thoughts?

pnutjam 5y ago
I'd say this depend on where you want to introduce the encryption and de-duplication. I'd lean towards 1, so each machine has a standalone encrypted backup, but 3 would provide easier access without the borg client for local backups, and better de-duplication if files are shared across machines.
fragileone 5y ago
I'd prefer 1, as it allows each machine to easily and quickly restore itself directly from a local backup. This way the ZFS and rsync.net backup components are only used incase of catastrophic failure.
infogulch 5y ago
If you have ZFS clients and server, what benefit does inserting borg provide on top of a continuous ZFS send from client -> server and server -> rsync.net? ZFS send can already do encryption, and ZFS supports compression. And snapshots automatically "dedups" files that haven't changed between snapshot points, which I'd guess saves more space than deduping literal duplicate files which are not very common ime. I guess I just don't see what value borg provides if you're already using ZFS throughout.
- indiv0 5y ago
  I guess my concern is that I'd like the data to be encrypted at rsync.net, which means I'd need to use raw ZFS send. Would it be possible to mount/browse those raw datasets remotely to verify my backups? Or would I need to recv it all back to be able to test them?

l1am0 5y ago

Love borg with the desktop client vorta. Gives my time-machine like time travel :D Even wrote a blog post about it: https://simon-frey.com/blog/borgvorta-is-finally-a-usable-backup-solution-for-linux-desktop/ https://simon-frey.com/blog/borgvorta-is-finally-a-usable-ba...

spaetzleesser 5y ago

Since I moved my personal laptop from Mac to Linux I have been using borg. It works really well so far. One thing that’s confusing me is the prune strategy. It seems prune removes the whole backup set at once. Let’s say I have daily backups and weekly backups. At some point I am pruning the daily backups. It seems this means that if the weekly backup ran on Sunday but a file was created on Monday and deleted on Friday, the file would be completely deleted during prune and there would be no trace of it. I would much prefer if the pruning was done on per file basis and not by pruning whole backup sets.

ThomasWaldmann 5y ago
Maybe have a look at borgbackup docs to use the same terminology, otherwise it might get confusing. When you create a backup ARCHIVE in a borg REPOSITORY, the archive contains all input files you gave to borg. Each archive has a name and usually the name is something like machinename-setname-date-time. You can create such backups rather often (that is cheap, due to deduplication), but you do not want to keep tons of archives long term (some borg commands take O(archive count) time). Thus, one runs borg prune to "thin out" archives and only keep some following some prune policy, like keeping 60 daily, 12 monthly, 10 yearly or so. It is important to use --prefix if one creates multiple different archive sequences in the same repo, e.g. if you create pc-home-date-time as well as pc-system-date-time - then you need to run prune with --prefix=pc-home- and another prune with --prefix=pc-system- . Usually, archives are not modified after their creation (the only way to do that is to carefully use borg recreate command).
axiolite 5y ago
> I would much prefer if the pruning was done on per file basis and not by pruning whole backup sets. Then just don't prune! Unchanged files take no space. There's no reason you can't keep many months of daily backups, and set your prune to only do: --keep-within=365d For those who aren't familiar with Borg, there is no such thing as a weekly backup, daily backup, monthly backup, etc. Every backup is a quick full backup. It's only the prune (which deletes old backups) option that gives you the choice of bracketing your backups into monthly/weekly/daily sets, so it'll save the last backup of each month for the last X months, the last backup of each week for X weeks, etc.