7 ms·

Also check out Borg: https://www.borgbackup.org https://www.borgbackup.org And some resources on how they're different: - https://github.com/restic/restic/iss

Also check out Borg: https://www.borgbackup.org https://www.borgbackup.org And some resources on how they're different: - https://github.com/restic/restic/issues/1875 https://github.com/restic/restic/issues/1875 - https://stickleback.dk/borg-or-restic/ https://stickleback.dk/borg-or-restic/ - https://sysadministrivia.com/episodes/S4E5 https://sysadministrivia.com/episodes/S4E5 The general concensus seems to be that restic is borg with more whistles (backing up to various places), but borg is the more trusted tool with the longer history (just use SSH and be done with it). I personally recently used borg for a migration between computers and it worked great for me.

noja 7y ago
restic is simpler than borg, because it doesn't require a special borg server (it just writes out to files - or block storage).
- Quekid5 7y ago
  This is important -- it means that restic can trivially be used with e.g. S3.
  - res0nat0r 7y ago
    I've loved using restic sofar. I've got about 3TB of stuff backed up to s3 using the restic cli. It is simple and just works. If anyone though knows how I can more easily restore a file that would be great. I have to supply the restoration snapshot ID right now, and I'd rather just do 'latest' and have it find the newest version of the file in all of the snapshots and restore it. Is this possible ? Something like: restic restore -i '/path/to/file.txt' -i . latest instead of: restic restore -i '/path/to/file.txt' -i . search-file-snapshot-list-for-big-sha-here
    - ailox 7y ago
      you could always mount the entire backup, using restic mount backup/ and pick the file(s) you are looking for from the `backup/snapshots/latest/` directory.
  - LeoPanthera 7y ago
    It also means that the sending client has to all the work. restic is enormously resource hungry compared to borg. It can be hit by the OOM killer on small VPSs.
- Erlich_Bachman 7y ago
  This is somewhat misleading: borg absolutely does not require its server software to be installed on the target server. It supports (and works well) with a simple sshfs mount.
derekp7 7y ago
There is also Snebu (www.snebu.com, Github repo at github.com/derekp7/snebu). I wrote this about 8 years ago for my personal needs, because rsync / snapshot backups were running out of steam for me. What's different from Restic or Borg/Attic, is that the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots, handles a number of systems easily, and is fairly fast due to using SQLite for the metadata catalog. A couple weeks ago I finished a client-side encryption module, got another weekend's worth of work to integrate encryption support into the back end. The client-side encryption module is already in the Git repository, but there is no documentation for it yet. My next project is a Web based GUI (maybe an Electron based client-side GUI too, depending on what makes sense). Oh, and the most recent release supports granular user permissions, so you can grant a host (via an SSH account on the backup server) permissions to create backups, but not delete them. Or have an administrative user that can expire old backups but not read backups, for example. Can help with thwarting crypto viruses that try to delete backups.
- brirec 7y ago
  > […] the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots […] So am I right to infer that there is a ridiculous amount of overhead in the data transfer, as opposed to some of the other software mentioned? Would you use this over the internet? Or an untrusted network connection?
  - derekp7 7y ago
    The communications is handled via ssh, so yes I would use it over the internet as the transfers are streaming. A full manifest (list of file names and metadata [file owner, mod time, size, etc]) is sent over the wire, and a return list of files that are needed to complete the snapshot (essentially changed / new files) is returned to the client. The client then tar's this smaller file list to stream to the remote server. Tar, in this case, is used to serialize the data, which gets extracted on the remote end (compressed and stored as sha1-named files [will be switching to sha-256 in the next major release]). Metadata gets put in the SQLite DB. For data encryption (the main part is completed, but need to expand the backend to recognize encrypted data, should be finished shortly) -- the output of "tar" is piped through "tarcrypt". What tarcrypt does is it takes a standard tar file input, compresses/encrypts the file data, and outputs a tar file with some extended headers that contain info about the compression/encryption, including the RSA public key fingerprint used to encrypt the data, an HMAC, and the encrypted (passphrase-protected) private key (this can be made optional). The idea is that the encryption itself is AES-256-GCM, with a random key, which is encrypted with RSA public key. That way you can have encrypted backups without needing to have a password sitting in plain text on the client. And the RSA private key is passphrase encrypted, and sent along with the tar header to the server. On restore, you will be prompted (client-side) for the passphrase. This way you can restore a client even if the keyfile is destroyed. I plan to make the encrypted key storage optional, but that would require that you manage the key file backup separately, and doesn't get you much more security (assuming you have an adequately strong passphrase). Server requirements are a server with ssh access, and the snebu binary installed (optionally suid to a non-privileged backup user account, so that granular permissions can be employed for other accounts). And since Snebu is written in C, with only liblzo2, libcrypt, and sqlite2 as dependencies, it is easy to get it to work with a wide variety of systems (and the client side only requires a modern enough version of GNU "find" and "tar", unless encryption is used, which would require "tarcrypt" also -- modern in this case means withing the last 10 years, the "find" command needs to support -printf with the appropriate parameters).
Osiris 7y ago
I use borg on a small VPS that hosts my side project to get free backups. Borg runs doing incremental backups to a local directory. I sync the borg backup folder to a free BackBlaze B2 bucket. The whole thing comes in around 5GB of backup including database backup and all the hosted files and configuration files.
Erlich_Bachman 7y ago
With the exception of being able to upload to different targets (which can be piggybacked onto borg with something like rclone), borg is definitely the one with more whistles, not the other way around. Restic is lagging behind borg in most features, they always come up in borg first and then after a year restic maybe implements them. Important features currently missing in restic which exist in borg: compression, append-only backups, extensive include/exclude patterns, flagging any directory for exclusion directory exclude with a tag file inside it, setting "--chunker-params" - thus being able to adjust deduplication settings (like size of chunks, how hard it tries to look for duplicates etc) to the nature of the files and the limitations of RAM and CPU during backup. (and other minor ones). Other differences which some might consider good and others bad include: * Borg is in Python, Restic is in Go? Python is a more widely known language, the chance that more developers will be able to maintain it is higher, the chance for new features is higher, the chance that bugs will be found is higher. Both still provide a static binary download so deployment is very easy for both. * Borg allows setting names for archives (which can still be templated with things like {now} or {username}), while Restic automatically names them with random ids. Borg's approach seems more user-friendly. * Borg allows more choice in what encryption is used, including authenticated/non-authenticated, and including using no encryption at all (might be useful in some cases). (https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-modes https://borgbackup.readthedocs.io/en/stable/usage/init.html#...) * Borg allows specifying "max_segment_size = xxxx" on a repository, thus allowing to somewhat adjust the size of archive ("segment") files while they are stored on the server (default being 500 MB). Restic uses its own sizing and (if I remember correctly) more often than not it produces very small files (a couple of MBs) which crowd the filesystem. In general, archive file sizes can affect the filesystem and the uploading them to the cloud, where many cloud providers API operates on individual files, not on parts of them.
- mike-cardwell 7y ago
  append-only backups: It supports this with the REST backend. flagging any directory for exclusion directory exclude with a tag file inside it: It has this. It's called --exclude-caches and will exclude any directory with a "CACHEDIR.TAG" file in with the correct content.
deleted 7y ago
[deleted]
blattimwind 7y ago
Borg's encryption is questionable at best. Performance wise it's not particularly good and probably will never get better because of the large, somewhat complex Python codebase.
- lima 7y ago
  What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way. I'm more concerned about the repository format and config file (i.e. attack surface, since the repo is potentially untrusted). Performance is actually better than Restic, and performance-critical parts of Borg are written in C or use C libraries.
  - blattimwind 7y ago
    > What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way. No, using one key per repository and a persistent message counter is not a reasonable design. https://borgbackup.readthedocs.io/en/stable/internals/security.html https://borgbackup.readthedocs.io/en/stable/internals/securi...
    - Erlich_Bachman 7y ago
      For what threat model does this matter?
      - blattimwind 7y ago
        "When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more)." Edit: I've posted this a bunch of times here, pretty much every time it caught my eye when someone said this tool has good crypto, and by now I'm used to people just downvoting it and saying it doesn't matter because obviously no one ever would use it like that and the design is fine etc. (isn't the point of deduplication to save disk space?)
- 7y ago
m3nu 7y ago
I would also say that Borg is more feature-complete and mature, not the other way round. If you don't want to run your own server, I offer a hosted Borg Backup service: https://www.borgbase.com https://www.borgbase.com This would give you some Borg-specific features, you can't easily get with your own VPS, like monitoring for outdated backups and restricting certain keys to append-only mode.

noja 7y ago

restic is simpler than borg, because it doesn't require a special borg server (it just writes out to files - or block storage).

Quekid5 7y ago
This is important -- it means that restic can trivially be used with e.g. S3.
- res0nat0r 7y ago
  I've loved using restic sofar. I've got about 3TB of stuff backed up to s3 using the restic cli. It is simple and just works. If anyone though knows how I can more easily restore a file that would be great. I have to supply the restoration snapshot ID right now, and I'd rather just do 'latest' and have it find the newest version of the file in all of the snapshots and restore it. Is this possible ? Something like: restic restore -i '/path/to/file.txt' -i . latest instead of: restic restore -i '/path/to/file.txt' -i . search-file-snapshot-list-for-big-sha-here
  - ailox 7y ago
    you could always mount the entire backup, using restic mount backup/ and pick the file(s) you are looking for from the `backup/snapshots/latest/` directory.
- LeoPanthera 7y ago
  It also means that the sending client has to all the work. restic is enormously resource hungry compared to borg. It can be hit by the OOM killer on small VPSs.
Erlich_Bachman 7y ago
This is somewhat misleading: borg absolutely does not require its server software to be installed on the target server. It supports (and works well) with a simple sshfs mount.

derekp7 7y ago

There is also Snebu (www.snebu.com, Github repo at github.com/derekp7/snebu). I wrote this about 8 years ago for my personal needs, because rsync / snapshot backups were running out of steam for me. What's different from Restic or Borg/Attic, is that the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots, handles a number of systems easily, and is fairly fast due to using SQLite for the metadata catalog. A couple weeks ago I finished a client-side encryption module, got another weekend's worth of work to integrate encryption support into the back end. The client-side encryption module is already in the Git repository, but there is no documentation for it yet. My next project is a Web based GUI (maybe an Electron based client-side GUI too, depending on what makes sense). Oh, and the most recent release supports granular user permissions, so you can grant a host (via an SSH account on the backup server) permissions to create backups, but not delete them. Or have an administrative user that can expire old backups but not read backups, for example. Can help with thwarting crypto viruses that try to delete backups.

brirec 7y ago
> […] the main "smarts" lives on the backup server (the Snebu binary), whereas the client side just uses find / tar to serialize the data. Backend side compresses, deduplicates, snapshots […] So am I right to infer that there is a ridiculous amount of overhead in the data transfer, as opposed to some of the other software mentioned? Would you use this over the internet? Or an untrusted network connection?
- derekp7 7y ago
  The communications is handled via ssh, so yes I would use it over the internet as the transfers are streaming. A full manifest (list of file names and metadata [file owner, mod time, size, etc]) is sent over the wire, and a return list of files that are needed to complete the snapshot (essentially changed / new files) is returned to the client. The client then tar's this smaller file list to stream to the remote server. Tar, in this case, is used to serialize the data, which gets extracted on the remote end (compressed and stored as sha1-named files [will be switching to sha-256 in the next major release]). Metadata gets put in the SQLite DB. For data encryption (the main part is completed, but need to expand the backend to recognize encrypted data, should be finished shortly) -- the output of "tar" is piped through "tarcrypt". What tarcrypt does is it takes a standard tar file input, compresses/encrypts the file data, and outputs a tar file with some extended headers that contain info about the compression/encryption, including the RSA public key fingerprint used to encrypt the data, an HMAC, and the encrypted (passphrase-protected) private key (this can be made optional). The idea is that the encryption itself is AES-256-GCM, with a random key, which is encrypted with RSA public key. That way you can have encrypted backups without needing to have a password sitting in plain text on the client. And the RSA private key is passphrase encrypted, and sent along with the tar header to the server. On restore, you will be prompted (client-side) for the passphrase. This way you can restore a client even if the keyfile is destroyed. I plan to make the encrypted key storage optional, but that would require that you manage the key file backup separately, and doesn't get you much more security (assuming you have an adequately strong passphrase). Server requirements are a server with ssh access, and the snebu binary installed (optionally suid to a non-privileged backup user account, so that granular permissions can be employed for other accounts). And since Snebu is written in C, with only liblzo2, libcrypt, and sqlite2 as dependencies, it is easy to get it to work with a wide variety of systems (and the client side only requires a modern enough version of GNU "find" and "tar", unless encryption is used, which would require "tarcrypt" also -- modern in this case means withing the last 10 years, the "find" command needs to support -printf with the appropriate parameters).

Osiris 7y ago

I use borg on a small VPS that hosts my side project to get free backups. Borg runs doing incremental backups to a local directory. I sync the borg backup folder to a free BackBlaze B2 bucket. The whole thing comes in around 5GB of backup including database backup and all the hosted files and configuration files.

Erlich_Bachman 7y ago

With the exception of being able to upload to different targets (which can be piggybacked onto borg with something like rclone), borg is definitely the one with more whistles, not the other way around. Restic is lagging behind borg in most features, they always come up in borg first and then after a year restic maybe implements them. Important features currently missing in restic which exist in borg: compression, append-only backups, extensive include/exclude patterns, flagging any directory for exclusion directory exclude with a tag file inside it, setting "--chunker-params" - thus being able to adjust deduplication settings (like size of chunks, how hard it tries to look for duplicates etc) to the nature of the files and the limitations of RAM and CPU during backup. (and other minor ones). Other differences which some might consider good and others bad include: * Borg is in Python, Restic is in Go? Python is a more widely known language, the chance that more developers will be able to maintain it is higher, the chance for new features is higher, the chance that bugs will be found is higher. Both still provide a static binary download so deployment is very easy for both. * Borg allows setting names for archives (which can still be templated with things like {now} or {username}), while Restic automatically names them with random ids. Borg's approach seems more user-friendly. * Borg allows more choice in what encryption is used, including authenticated/non-authenticated, and including using no encryption at all (might be useful in some cases). (https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-modes https://borgbackup.readthedocs.io/en/stable/usage/init.html#...) * Borg allows specifying "max_segment_size = xxxx" on a repository, thus allowing to somewhat adjust the size of archive ("segment") files while they are stored on the server (default being 500 MB). Restic uses its own sizing and (if I remember correctly) more often than not it produces very small files (a couple of MBs) which crowd the filesystem. In general, archive file sizes can affect the filesystem and the uploading them to the cloud, where many cloud providers API operates on individual files, not on parts of them.

mike-cardwell 7y ago
append-only backups: It supports this with the REST backend. flagging any directory for exclusion directory exclude with a tag file inside it: It has this. It's called --exclude-caches and will exclude any directory with a "CACHEDIR.TAG" file in with the correct content.

deleted 7y ago

[deleted]

blattimwind 7y ago

Borg's encryption is questionable at best. Performance wise it's not particularly good and probably will never get better because of the large, somewhat complex Python codebase.

lima 7y ago
What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way. I'm more concerned about the repository format and config file (i.e. attack surface, since the repo is potentially untrusted). Performance is actually better than Restic, and performance-critical parts of Borg are written in C or use C libraries.
- blattimwind 7y ago
  > What's questionable about it? It seems that they use modern AEAD ciphers in a reasonable way. No, using one key per repository and a persistent message counter is not a reasonable design. https://borgbackup.readthedocs.io/en/stable/internals/security.html https://borgbackup.readthedocs.io/en/stable/internals/securi...
  - Erlich_Bachman 7y ago
    For what threat model does this matter?
    - blattimwind 7y ago
      "When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more)." Edit: I've posted this a bunch of times here, pretty much every time it caught my eye when someone said this tool has good crypto, and by now I'm used to people just downvoting it and saying it doesn't matter because obviously no one ever would use it like that and the design is fine etc. (isn't the point of deduplication to save disk space?)
7y ago

m3nu 7y ago

I would also say that Borg is more feature-complete and mature, not the other way round. If you don't want to run your own server, I offer a hosted Borg Backup service: https://www.borgbase.com https://www.borgbase.com This would give you some Borg-specific features, you can't easily get with your own VPS, like monitoring for outdated backups and restricting certain keys to append-only mode.