20 ms·

BorgBackup: Deduplicating archiver with compression and encryption

334 points by phil294 4y ago

sleepytimetea 4y ago
Python source code ? No cloud native API integration ? UI?
- non-nil 4y ago
  There's Vorta: https://github.com/borgbase/vorta https://github.com/borgbase/vorta which I quite like.
  - eointierney 4y ago
    Deffo recommend Vorta, good ui, very reliable
  - eternityforest 4y ago
    Vorta looks really awesome, maybe awesome enough that I might switch from Back in Time.
    - SoylentYellow 4y ago
      I made the switch! Vorta works very well.
  - jrootabega 4y ago
    Hey, you can't just mix quadrants like that.
  - teejee 4y ago
    If you don't mind a paid app, there's also Baqpaq (written by me) https://teejeetech.com/tag/baqpaq/ https://teejeetech.com/tag/baqpaq/
- ThomasWaldmann 4y ago
  Besides vorta (which offers a lot of options) there is also pika backup (which might be simpler by offering less options).
- ThomasWaldmann 4y ago
  Python source code? Yes, it is on https://github.com/borgbackup/borg https://github.com/borgbackup/borg (if that was the question). No cloud? Indeed, borg does not integrate cloud functionality. You'll either need a directory or a server with ssh and borg to store your backups into. Some people use rclone to clone their borg repos to the cloud.
samuell 4y ago
Have had a look at both Borg and Restic, but even Restic which is supposed to be faster iirc, was extremely slow on my computer. Been much more happy with my tries with https://kopia.io https://kopia.io which also includes an optional cross-platform GUI, in addition to the CLI.
- beci 4y ago
  I tried all these in my environment, about 2 years ago and kopia wins for me too. Is there any advantage of borg over kopia since then?
  - aborsy 4y ago
    Reliability. Borg has been around for a long time, and is far more mature. I wouldn’t trust my backups to Kopia (unless for experimentation).
- nine_k 4y ago
  Wow, Kopia looks pretty interesting. I first thought that it's a fork of Restic, but it appears to be independent. It has all the features that are key to me: encrypted, deduplicated, works on object storage, can mount the backup as a filesystem. On one hand, on may think that three programs with very similar approach and features is a waste of resource. On the other hand, this is what refinement of the idea looks like: each project improves over previous attempts.
wyatt_dolores 4y ago
I had to setup a quick backup to s3 storage to replace an aging rsnapshot setup. I looked at Borg, but Duplicity (https://duplicity.us/ https://duplicity.us/) was easier to configure and connect to S3. For syncing S3 storage across providers, I went with rclone (https://rclone.org/ https://rclone.org/). Note that using rclone to sync across providers (e.g. from Amazon to Wasabi) does require the files to be downloaded to the client machine and then uploaded again. Not ideal, but if you have extra bandwidth it is a convenient setup.
- ThomasWaldmann 4y ago
  I see quite some full backups in your near future. ;-) And that is one of the main reasons why chunk-deduplicating backup tools (like borg, restic, ...) are better than full/incremental style ones.
  - jrootabega 4y ago
    Is there really a difference between deduplication and an infinite chain of incrementals, in this respect? Other than the restore process?
    - ThomasWaldmann 4y ago
      deduplicating backup tools use a symmetric approach: all archives needing some chunk (piece of data) reference it via its chunkid (the keyed hash over the chunk plaintext content). so you can delete ANY backup archive without influencing any other backup archive. a chunk will be only deleted if nothing is referencing it any more. also, each backup is logically a FULL backup (it has ALL files, references ALL content data). it is just made in a clever way, avoiding to re-transfer data that already is present in the backup repository, thus it FEELS like incremental (considering speed, amount of CPU and I/O used). OTOH, full/incremental style backup tools build a chain of incremental archives depending on the previous incremental and the full backup, which gets more fragile the longer the chain gets. because of that and also because you might want to delete older backups at some time, you are forced to create new full backups regularly (causing lots of CPU and I/O load).
bluedino 4y ago
Borg works great. I used it at a shop that had lots of servers, but didn't have any real backups-other than when someone would remember to go swap external USB drives and hope they actually ran. Set it up on a bunch of servers with a simple cron job, the initial backups went quickly, and the incrementals were really fast. Made great use of an old Dell server that wasn't doing anything else and had lots of slow disks in it.
- m3nu 4y ago
  Borg is surprisingly fast and memory-efficient, even when compared to Restic, which is written in Go. Recently did a benchmark to test the upcoming Borg v2 and this surprised me the most: https://github.com/borgbase/benchmarks https://github.com/borgbase/benchmarks
  - number6 4y ago
    I am always torn between the two: restic or Borg.. how would I decide?
    - water554 4y ago
      Use them both I ended up at borg
trulyrandom 4y ago
I've been using Borg for years. It's great! The deduplication feature allows me to take a "full" backup of my work station hourly. Taking frequent backups like this has already saved my bacon a number times in cases where I accidentally mangled/deleted a file I didn't mean to touch. I recently stumbled upon the release notes for the (WIP) v2: https://www.borgbackup.org/releases/borg-2.0.html https://www.borgbackup.org/releases/borg-2.0.html. Seems to address quite a few of the pain points of v1.
- BrotherBisquick 4y ago
  I wonder what exactly will happen when v2 hits the Arch main repo. Will my automated hourly backups (via the Borgmatic wrapper) just start throwing errors and stop working?
- Derbasti 4y ago
  Same here, using it for years! The deduplication allowed me to move from Windows to Linux, while keeping a consistent backup. It's great stuff!
- m3nu 4y ago
  Yepp. Version 2 got rid of lots of legacy code and cleaned up CLI args a bit. Will be around 5 to 20% faster than the v1.2 branch. https://github.com/borgbase/benchmarks https://github.com/borgbase/benchmarks
  - mustache_kimono 4y ago
    Yeah, last time I tried, it was impressive, but kinda slow and limited to execution on a single core. Eager to take another look at Borg and Kopia, etc.
    - unaindz 4y ago
      Take a look into bupstash, multithreaded and really fast.
- beci 4y ago
haunter 4y ago
What’s good for Windows 10 (NTFS) drives? I’m using the Veeam Agent free version [0] for years and no problems whatsoever but curious what are some good options 0, https://www.veeam.com/agent-for-windows-community-edition.html https://www.veeam.com/agent-for-windows-community-edition.ht...
- xupybd 4y ago
  I use restic on Windows servers but for workstations I use backblaze. They have a backup client. It's just too easy. I don't have to think about it.
- k8sToGo 4y ago
  For Windows Image Backups I use macrium
- MikusR 4y ago
  Macrium for system images and Kopia for files
- michaelcampbell 4y ago
  Restic works on windows.
mtmail 4y ago
rsync.net has a special discount when you use borg and "you're an expert" https://www.rsync.net/products/borg.html https://www.rsync.net/products/borg.html We're looking to replace our self-written borg backup scripts with https://torsion.org/borgmatic/ https://torsion.org/borgmatic/ which is a wrapper around borg.
- n8henrie 4y ago
  TIL! Same for restic: https://www.rsync.net/products/restic.html https://www.rsync.net/products/restic.html
  - stavros 4y ago
    As far as I can tell (as a years-long user of this), this is a generic "here's a server that supports SCP, go nuts" discount. You don't have to use it with Borg or Restic, they're just supported.
    - asymmetric 4y ago
      Notably, you can't reformat the drives to another filesystem (say btrfs)
      - stavros 4y ago
        Can you do that over SCP in general, without a full shell?
- berkut 4y ago
  What are the practicalities of the '680 GB Minimum Order'? I would have expected you pay by how much you use (hence the price per GB), but the minimal order (the submission form however won't let you submit less than 100GB, so it's a bit confusing there's a mis-match?) seems slightly at odds with that. (The FAQ doesn't seem to answer that)
  - andreareina 4y ago
aborsy 4y ago
Borg is very good. The V2 repository format will bring in a lot of improvements, particularly in cryptography. Anyone knows when 2.0 will be out of beta, and stable?
- m3nu 4y ago
  Likely next year after 1-2 RCs. It's at beta4 currently.
nine_k 4y ago
If you prefer a similar approach, but as a single compiled binary, there's Restic: https://github.com/restic/restic https://github.com/restic/restic Update: yet another take on basically the same approach, also as a self-contained binary: https://github.com/kopia/kopia https://github.com/kopia/kopia
- kova12 4y ago
  I recall trying to use restic instead of borg a couple years ago, and some major feature was unavailable. I don't recall what is was, I think it was compression, which made archives quite large, and required larger instances for backup.
  - deleted 4y ago
    [deleted]
  - anotherevan 4y ago
    It probably was compression. The good news is compression is now available with Restic!
- RockRobotRock 4y ago
  Does anyone have a take on Kopia vs Restic?
  - btschaegg 4y ago
    A major factor why I wouldn't want to use Kopia (I looked into it) ist that it is opinionated with regards to how your system is set up (old-school unix FS layouts in a "pets, not cattle" way). It assumes the location of config files and does not allow you to change the backup path that's stored in a snapshot's metadata. That's bad if you want to use it 1) on NixOS (I don't want backup configs laying around in `~/.config`). As Indy famously said: "That belongs in a Nix expression!" Edit: On a second note: Why have mandatory config files at all? I don't have an issue with having the option or it being the default, but for my use case being able to specify the whole repository config via arguments sounds considerably more sane. 2) with ZFS snapshots (yes, I'm backing up `/path/to/dataset/.zfs/snapshot/<timestamp>/foo/bar`, but that should not be its path in the metadata!) OTOH, it seems to have the upside that you can apparently alter snapshots after the fact more easily (e.g. if you find out you shouldn't have backed up that gigantic VM image you just moved somewhere temporarily). I leave the decision on whether this is a footgun or not to you. And to be clear: The ZFS snaphot thing is also a pain with Restic, too. You can hack around it somewhat better with something like systemd-nspawn, but it really shouldn't be that hard.
SoftTalker 4y ago
Maybe a bit off topic, but what is a good utility for "imaging" a linux system. I have a task to reprovision a system but we want to keep a complete backup of the current system so that it's possible to restore completely as if it were never touched. This is more than just data backup as we would need need to recover disk partitions/LVM metadata, boot records, etc. as well as all the data itself.
- av8avenger 4y ago
  Take a look at Clonezilla. Used it many times for the exact same purpose. You could run it either on a running system or use the live iso they provide. https://clonezilla.org/ https://clonezilla.org/
- eointierney 4y ago
  Clonezilla is awesome, fast, stable, flexible, and reliable, from the Taiwan Supercomputing Centre. I used it lots over a decade ago to manage Mac, Windows, and Linux workstations and servers.
- akerl_ 4y ago
  Do you need to do this once, or 10 times, or 1000 times? How big are the disks? The most boring answer is "connect the disks to something else and use `dd` to copy the full blocks from start to finish into a file".
  - SoftTalker 4y ago
    For this specific need, just once. Disk is 1TB but only about 350GB used. 'dd' would have been my thought as well, I've heard of Clonezilla also but never used it and not sure it's really doing anything appreciably different. I like the idea of 'dd' because I have a very clear mental picture of what it does. Just wasn't sure there was something else I might want to look at.
    - dividuum 4y ago
      If you need multiple version of such a disk image, a tool like restic (or I guess borg too, not sure?) can also compress what's provided to it via stdin. So you'd dd directly into restic and it will delta compress to earlier backups.
vbezhenar 4y ago
Can someone suggest an approach to backup container environment? E.g. running inside Kubernetes. As I see it: I write some kind of configuration. someproject-db is a deployment which runs a postgres db. Tool should connect to this DB, issue some kind of pg_backup command, capture output, retrieve some metadata about previous backup from S3, compute difference with previous run, compress that difference and store it to S3. anotherproject is a deployment which runs an sqlite db. Tool should do the same but with sqlite-specific commands. yetanotherproject-data is a pvc which has attached pv. Tool should find pod which mounted this volume, exec into that pod and retrieve pv data, again find different and store it to S3. Of course things should be configurable. Like store difference every 15 minutes, store complete backup every week and so on. I'm fine with manual recover and with manual configuration (I just don't want to write and test all the scripts myself). What I don't want is some kind of magic tool which will backup the entire cluster, etcd and my grandparents automatically in some magic way only for $50k/cpu core.
- aborsy 4y ago
  Database and VM snapshot and backup can be tricky. My suggestion is using ZFS.
  - mekster 4y ago
    Doesn't zfs kind of solve all the backup problems alone? Technically, no other backup tools can beat it as being the filesystem itself, it knows more than any external tools can know, like instantly know what file got changed over time without scanning the entire tree. I use Borg as backup of backup (zfs snapshots), so I'll be having multiple implementations of backups (also both are on different remote location) just to be on the safe side. I don't use any other fancier ones as I don't like risking data on less reliable tools.
    - m3nu 4y ago
      Remote ZFS replication kinda does. But the offsite backup wouldn't be encrypted and not everyone is using ZFS. So it's not for all situations.
dang 4y ago
Related: Deduplicating Archiver with Compression and Encryption - https://news.ycombinator.com/item?id=27939412 https://news.ycombinator.com/item?id=27939412 - July 2021 (71 comments) BorgBackup: Deduplicating Archiver - https://news.ycombinator.com/item?id=21642364 https://news.ycombinator.com/item?id=21642364 - Nov 2019 (103 comments)
ThePhysicist 4y ago
Have been using Borg for many years now, it saved me several times already when I accidentally deleted stuff I realized I still needed later on. What's great is that you can just mount your backup repository as a FUSE filesystem, Borg then gives you a directory structure containing all your backups over time. Personally I use dates to name my backups, e.g. 2022-10-11, so when I need to restore something from a specific date I just go to the appropriate folder and extract it.
- mgbmtl 4y ago
  Another nice thing with fuse is that I can let my users (devs) mount the backups with Borg, so that they can restore specific files that they need without asking me. And the mount can restrict which directories to expose, so they only see the files they need. (I wrote a small wrapper which can be run with sudo)
andrewchambers 4y ago
I am the author of bupstash - https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash which has many advantages over borg in my biased opinion (like air gapped decryption keys and better performance). Feel free to check it out.
- g_p 4y ago
  Thanks for your work on bupstash. To second this, there are some threat models that are often overlooked with backups. Separating the encryption and restore keys makes sense, and is a design pattern few of the modern new backup tools offer - bupstash and duplicacy are the two that spring to mind which support asymmetrically encrypted backups. The recent lastpass issue shows the importance of keeping the backup decryption keys offline and inaccessible. A headless server with backup script can upload backups, but if someone can access the script and backups, they have the golden goose (users' vaults). Most other backup tools don't give you the ability to separate out creating and retrieving backups. Separate moan - very few of the backup tools that use s3 style buckets give a proper description of the IAM security model you should use - I like to understand what object prefixes are used, and what the absolute minimum I require is, as I like to start from minimum privilege - PutObject, GetObject on an index or config file, and nothing else if I can avoid it! Hopefully we see more focus on backup security after lastpass.
- dvzk 4y ago
  I was about to mention this. It’s hard for me to take any symmetric or password-based backup tools seriously. User chosen passwords are a disaster for offline encryption. Most tools also use related key input for (non-)authenticated encryption and decryption. A server shouldn’t be able to decrypt backups and authentically encrypt new backups.
- arde 4y ago
  Very interesting, I like its design better. Could you comment on whether bupstash supports ACLs (POSIX, xattr, etc.)? According to issue #25 it does, but I couldn't find documentation on how to exclude it when restoring, viewing it, etc.
m3drano 4y ago
I recommend using borgmatic to ease the management of Borg backups.
nov21b 4y ago
Just started using the append only feature to prevent a potential hacker from wiping out backups that live on a remote ssh server. Combined with restricted ssh access this can be made quite secure. I also tested writing backups to my Android phone (as a backup target) using Termux and Wireguard, worked flawlessly with a bit of tuning (keeping the vpn alive)
- pkulak 4y ago
  Append only modes are brilliant. Is there an easy way to hook into something like Glacier Deep Archive? That would be super cost effective.
  - nov21b 4y ago
    Not without akward hacks. [0] I backup to a Hetzner storage box and a Raspberry Pi at home. [0] https://github.com/borgbackup/borg/issues/1872 https://github.com/borgbackup/borg/issues/1872
lyu07282 4y ago
Its pretty great, I just wished it somehow integrated with rclone, I wish I could directly backup to google drive with it.
- beagle3 4y ago
  restic has a very similar feature set (it has compression now), and is well integrated with rclone.
tschuehly 4y ago
I'm searching for an alternative to OneDrive for my personal files, as I'm already paying for 1TB S3 Storage I wanted to use that as Backend. What tool could I use that has a windows client, android and ios?
phyzome 4y ago
I've been very happy with it. I switched away from Tarsnap because it was far too slow on restores. BorgBackup has slightly different security semantics, but similarly supports the notion of append-only credentials. And borgbase.com has been a good place to host my backups. Really nice onboarding flow.
- lighttower 4y ago
  I've been using it with Rsync.net also very happy.
- andreareina 4y ago
  Borg has an append-only mode, but no append-only credentials as far as I could find (by necessity given the symmetric encryption I believe). That means the key still needs to be protected, which makes secure automated backups a bit of a pickle.
  - g_p 4y ago
    If you use a remote Borg server over SSH, you can set up your authorized_keys file to enforce append-only mode on the server by defining the command to be run at login, and reject Borg requests by that SSH key which try to purge or remove data. I believe this is roughly what borgbase does to implement their own backup protection features.
    - andreareina 4y ago
      Yeah I suppose that's a fair interpretation of the authorized_keys setup.
  - phyzome 4y ago
    Ah, yes, my apologies -- it's done via a connection mode.
dawnerd 4y ago
Borg is really solid and I had to unfortunately use it to restore data a bit ago and it went very smoothly. I’m bad with testing backups so I was a bit nervous.
gorgoiler 4y ago
I’ve been using Borg for the last three years to replicate on-site ZFS snapshots as Borg archives on rsync.net. My local ZFS is plaintext on LUKS volumes and the remote Borg archives are encrypted with borg. Would I be better off using ZFS native encryption and replicating the encrypted stream without giving a key to the rsync.net side? I worry that one false move would mean I would have to do a full backup again. I think I kind of feel more comfortable having borg as a second technology in the system, too, so that my restore can’t be broken by a ZoL bug. Doing test restores with Borg is also nice because I don’t need a ZFS system to do the restore — restoring to macOS works just fine for example. Funnily enough, I’ve never had to prune / ruler function any Borg archives. I just don’t have enough churn in my data to need to do it. Everything change is there in full. It works so well. Thank you everyone involved for donating your time and ideas to providing these tools and services.
- rsync 4y ago
  "Would I be better off using ZFS native encryption and replicating the encrypted stream without giving a key to the rsync.net side?" I think you mean you are using borgs "repo key" method where the key to the repo is stored, remotely, inside the repo, correct ? Just to clarify for others: That repo-stored key has a passphrase so this method still keeps all of your files encrypted in a way that nobody at rsync.net can access. FWIW, I use this method personally - I like the idea of storing the key with the repo and unlocking it with a passphrase.
  - gorgoiler 4y ago
    Thanks for responding and for providing such a great service. I meant dropping borg altogether and using zfs send/receive to sync my data to rsync.net. As I understood it, if my dataset is encrypted then rsync.net only sees the name of the snapshot and the encrypted data. I’m not sure this would be a great solution for my most common restore use case though: fetching a single file back with borg while I’m at a third location that is remote to both rsync.net and my ZFS host. The fact that I can do that with just Borg and rsync.net is incredibly useful (yes, I travel with my keys!)
unaindz 4y ago
I tried a few backup tools and https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash is my favorite by far but it's not that well known. It was pretty fast already and recently got multithread support. It has been the only thing usable for backing up a few TB in a raspberry for performance reasons. Keep in mind it's relatively new and the author does not yet recommend to use in production as the only backup solution.
- nextaccountic 4y ago
  does this also backup mtime and ctime timestamps?
  - andrewchambers 4y ago
    Yes and it can restore mtime and atime - but not ctime.
    - nextaccountic 4y ago
      What can it do with ctime? Maybe it could provide a virtual read only filesystem, mounting the backup at a specific date, using the original atime, mtime and ctime
      - andrewchambers 4y ago
        It can provide diffs against ctime so you can see if files have been modified and then restored.
layer8 4y ago
Can anyone recommend a (Linux CLI) backup tool for incremental encrypted backups via SCP that allows a verification run against the current filesystem state while excluding a configured list of paths from the verification (for some frequently changing files)? I’m currently using duplicity, which is missing that last feature (without post-processing its verification output).
- stavros 4y ago
  Restic can do all that, I believe.
  - deleted 4y ago
    [deleted]
mikob 4y ago
How does this compare to rsnapshot?
- ThomasWaldmann 4y ago
  Not much. rsnapshot is based on rsync and hardlinks. So, you'll get some whole-file deduplication for unmodified files. No compression, no encryption/authentication, no way to verify if a backup is still ok. Also, if you rename a file or folder, that hardlink-based dedup will not work, because it is based on files having the same path. Also, if you change just 1 bit in a huge file, it will add the whole new file because the dedup only works on whole files. borg does variable size chunk based deduplication. if a big file changed just a bit, your backup repo will also just grow a bit when backing up that file. also you get compression, encryption, authentication and the ability to check backups if they are ok still. also, borg won't create a gazillion of files/hardlinks in the destination filesystem, but way fewer files (each about 500MiB big by default). borg also archives ACLs, xattrs, flags.
tut-urut-utut 4y ago
> Easy installation on multiple platforms: Linux, macOS, BSD, ... No Windows :( Luckily there are capable alternatives to Borg, including Restic and Kopia.
bilekas 4y ago
Hmmm I like this a lot actually, but as an encrypted file repo for sensitive files/repos themselves. This is quite clever, don't think I've seen all these features together before.
Brian-Puccio 4y ago
I’ve been using BorgBackup since it was Attic. Been thrilled with it.
siraben 4y ago
Absolutely love Borg, have been using it for several years. Cross-platform (macOS and Linux) backups, deduplication, encryption and compression, FUSE mounting, backing up over SSH (works great in combination with Tailscale and can be interrupted/resumed). Pruning backups with rules is also easy, and sometimes I accidentally back up really large folders in which case Borg lets you delete it from all the backups.
avodonosov 4y ago
Is Borg significantly better than rsync to a btrfs or zfs volume, and then snapshotting that volume?
pQd 4y ago
borg is great. we've been using it for the past 3 years to archive hundreds of file-level backups of servers, database dumps and VM images. average size of each borg repo is few GB but there are few outliers up to few hundreds of GB. most of backups are done daily, with 7-24 past days preserved in borg archive. borg repos are verified, copied to external disks, verified again and rotated offline each week. borg replaced https://rdiff-backup.net/ https://rdiff-backup.net/ for us and gave: * nice speedup of backups/backup tests, * decent saving in the disk space thanks to compression and deduplication, * decreased backup replication time [ borg repo tends to have much less, larger files compared to rdiff which has in its repo at least as many files as your source data; rsync likes it ]. to finish backups in a reasonable time we had to parallelize backup gathering [ each server / vm goes to separate borg repo; this limits a failure domain in case of corrupted repo, but denies us benefit of deduplication on larger scale - across servers ] and borg archiving. without that - we would be a limited by a single cpu core performance [ borg is not multithreaded yet ]. it's worth testing the backups - we're doing it each day by using borg's repo self test and by extracting few key files and checking their checksums and content... just in case. echoing other comments - https://kopia.io/ https://kopia.io/ looks interesting but we have not tried it yet.
AnonC 4y ago
Any pointers on running borg or restic on a Synology NAS running DSM 7 (to backup the NAS to another destination)? The Synocommunity package for Borg generally tends to be lagging behind the official releases by months. And Docker seems to be a bit complex too (since the NAS I have directly doesn’t support it).
leaving 4y ago
I don't understand why creators of projects like this obscure the basics. Important things you need to tell me: (1) The file I download is a command-line executable. I have to chnmod +x before I can use it. Why not share that small fact? (2) How do I create a simple repository *with no encryption*. Yes, I want to start there. Why do you think I want to get a PhD in your project before trying it? No, I don't want to start by creating encryption keys; that comes later. Start with the basic, basic, basic instructions. (3) Why did I export that environment variable? I never used it, did I? Not so far as I can tell. Is that basic? No, it is not. (4) What does recreate do? Is it necessary? "borg rcreate --help" does *not* provide any help on the encryption options. It just provides an error. Seriously, what is wrong with you people? Can you not imagine what it is like to come into an open source project with no experience and no knowledge of the matter whatsoever? Is it machismo? Is it elitism? What, in the name of God, prevents you from saying, "This is my project. It does this. This is the "hello world" method of trying it. Here are some cool options once you have the basics. It happens almost all the time. I seriously don't know what is wrong with you. Edit: I had to search forever to find the keyword "none" that you can put after the "-e" parameter. Found it by trial and error. It's in the docs, but, Jesus, people, can you not imagine for a moment that someone might be testing this on not the latest hardware and want to see how it runs without the complexity encryption first? You absolutely blow my mind. I don't know if you're putting me on, here. Do you not know how to communicate?
- dikei 4y ago
  You're welcome to read the Quick Start docs, it's pretty short https://borgbackup.readthedocs.io/en/1.2-maint/quickstart.html https://borgbackup.readthedocs.io/en/1.2-maint/quickstart.ht...
- crossroadsguy 4y ago
  I agree. Borg (and restic as well - though it fares better than borg in usability), even after tinkering with it for years, remained essentially useless for me until Vorta (a borg gui) came along. https://vorta.borgbase.com https://vorta.borgbase.com is awesome. Simple and awesome. I wish it gains more exposure.
- sigtstp 4y ago
jve 4y ago
Dunno if this 1 year old comment is relevant, but to whomever it may concern, it may be useful to research backup software stability & reliability: https://news.ycombinator.com/item?id=29210222 https://news.ycombinator.com/item?id=29210222
asymmetric 4y ago
Is there anything like zfs.rent for btrfs? I would like to use my filesystem’s facilities for backing rather than a whole piece of software on top, but I haven’t found any managed storage solutions yet.
INTPenis 4y ago
I used borg for years, never had any issue. Even though there were scares of bad versions that could corrupt your archives. Never happened to my personal backups of a few hundred Gs at most. But I had to switch to restic because the S3 support was too good. Off-site backups in one command, instead of having to maintain my borg repos on NAS and external disks.
potamic 4y ago
Can you use this for a full system backup or is it more suited towards backing up individual files?
qwertygerty 4y ago
Unlike some of the negative comments seen, I'd like to extend my gratitude to the developers for this work! I've recently decided to review my backup scheme, and have been looking at btrfs. Anyone out there doing a btrfs+borg combo? What is your scheme? I've found this as an example, but it's quite a bit more intense than what I need... https://mutschler.dev/stuff/backup/ https://mutschler.dev/stuff/backup/

334 points by phil294 4y ago

sleepytimetea 4y ago

Python source code ? No cloud native API integration ? UI?

non-nil 4y ago
There's Vorta: https://github.com/borgbase/vorta https://github.com/borgbase/vorta which I quite like.
- eointierney 4y ago
  Deffo recommend Vorta, good ui, very reliable
- eternityforest 4y ago
  Vorta looks really awesome, maybe awesome enough that I might switch from Back in Time.
  - SoylentYellow 4y ago
    I made the switch! Vorta works very well.
- jrootabega 4y ago
  Hey, you can't just mix quadrants like that.
- teejee 4y ago
  If you don't mind a paid app, there's also Baqpaq (written by me) https://teejeetech.com/tag/baqpaq/ https://teejeetech.com/tag/baqpaq/
ThomasWaldmann 4y ago
Besides vorta (which offers a lot of options) there is also pika backup (which might be simpler by offering less options).
ThomasWaldmann 4y ago
Python source code? Yes, it is on https://github.com/borgbackup/borg https://github.com/borgbackup/borg (if that was the question). No cloud? Indeed, borg does not integrate cloud functionality. You'll either need a directory or a server with ssh and borg to store your backups into. Some people use rclone to clone their borg repos to the cloud.

samuell 4y ago

Have had a look at both Borg and Restic, but even Restic which is supposed to be faster iirc, was extremely slow on my computer. Been much more happy with my tries with https://kopia.io https://kopia.io which also includes an optional cross-platform GUI, in addition to the CLI.

beci 4y ago
I tried all these in my environment, about 2 years ago and kopia wins for me too. Is there any advantage of borg over kopia since then?
- aborsy 4y ago
  Reliability. Borg has been around for a long time, and is far more mature. I wouldn’t trust my backups to Kopia (unless for experimentation).
nine_k 4y ago
Wow, Kopia looks pretty interesting. I first thought that it's a fork of Restic, but it appears to be independent. It has all the features that are key to me: encrypted, deduplicated, works on object storage, can mount the backup as a filesystem. On one hand, on may think that three programs with very similar approach and features is a waste of resource. On the other hand, this is what refinement of the idea looks like: each project improves over previous attempts.

wyatt_dolores 4y ago

I had to setup a quick backup to s3 storage to replace an aging rsnapshot setup. I looked at Borg, but Duplicity (https://duplicity.us/ https://duplicity.us/) was easier to configure and connect to S3. For syncing S3 storage across providers, I went with rclone (https://rclone.org/ https://rclone.org/). Note that using rclone to sync across providers (e.g. from Amazon to Wasabi) does require the files to be downloaded to the client machine and then uploaded again. Not ideal, but if you have extra bandwidth it is a convenient setup.

ThomasWaldmann 4y ago
I see quite some full backups in your near future. ;-) And that is one of the main reasons why chunk-deduplicating backup tools (like borg, restic, ...) are better than full/incremental style ones.
- jrootabega 4y ago
  Is there really a difference between deduplication and an infinite chain of incrementals, in this respect? Other than the restore process?
  - ThomasWaldmann 4y ago
    deduplicating backup tools use a symmetric approach: all archives needing some chunk (piece of data) reference it via its chunkid (the keyed hash over the chunk plaintext content). so you can delete ANY backup archive without influencing any other backup archive. a chunk will be only deleted if nothing is referencing it any more. also, each backup is logically a FULL backup (it has ALL files, references ALL content data). it is just made in a clever way, avoiding to re-transfer data that already is present in the backup repository, thus it FEELS like incremental (considering speed, amount of CPU and I/O used). OTOH, full/incremental style backup tools build a chain of incremental archives depending on the previous incremental and the full backup, which gets more fragile the longer the chain gets. because of that and also because you might want to delete older backups at some time, you are forced to create new full backups regularly (causing lots of CPU and I/O load).

bluedino 4y ago

Borg works great. I used it at a shop that had lots of servers, but didn't have any real backups-other than when someone would remember to go swap external USB drives and hope they actually ran. Set it up on a bunch of servers with a simple cron job, the initial backups went quickly, and the incrementals were really fast. Made great use of an old Dell server that wasn't doing anything else and had lots of slow disks in it.

m3nu 4y ago
Borg is surprisingly fast and memory-efficient, even when compared to Restic, which is written in Go. Recently did a benchmark to test the upcoming Borg v2 and this surprised me the most: https://github.com/borgbase/benchmarks https://github.com/borgbase/benchmarks
- number6 4y ago
  I am always torn between the two: restic or Borg.. how would I decide?
  - water554 4y ago
    Use them both I ended up at borg

trulyrandom 4y ago

I've been using Borg for years. It's great! The deduplication feature allows me to take a "full" backup of my work station hourly. Taking frequent backups like this has already saved my bacon a number times in cases where I accidentally mangled/deleted a file I didn't mean to touch. I recently stumbled upon the release notes for the (WIP) v2: https://www.borgbackup.org/releases/borg-2.0.html https://www.borgbackup.org/releases/borg-2.0.html. Seems to address quite a few of the pain points of v1.

BrotherBisquick 4y ago
I wonder what exactly will happen when v2 hits the Arch main repo. Will my automated hourly backups (via the Borgmatic wrapper) just start throwing errors and stop working?
Derbasti 4y ago
Same here, using it for years! The deduplication allowed me to move from Windows to Linux, while keeping a consistent backup. It's great stuff!
m3nu 4y ago
Yepp. Version 2 got rid of lots of legacy code and cleaned up CLI args a bit. Will be around 5 to 20% faster than the v1.2 branch. https://github.com/borgbase/benchmarks https://github.com/borgbase/benchmarks
- mustache_kimono 4y ago
  Yeah, last time I tried, it was impressive, but kinda slow and limited to execution on a single core. Eager to take another look at Borg and Kopia, etc.
  - unaindz 4y ago
    Take a look into bupstash, multithreaded and really fast.
beci 4y ago

haunter 4y ago

What’s good for Windows 10 (NTFS) drives? I’m using the Veeam Agent free version [0] for years and no problems whatsoever but curious what are some good options 0, https://www.veeam.com/agent-for-windows-community-edition.html https://www.veeam.com/agent-for-windows-community-edition.ht...

xupybd 4y ago
I use restic on Windows servers but for workstations I use backblaze. They have a backup client. It's just too easy. I don't have to think about it.
k8sToGo 4y ago
For Windows Image Backups I use macrium
MikusR 4y ago
Macrium for system images and Kopia for files
michaelcampbell 4y ago
Restic works on windows.

mtmail 4y ago

rsync.net has a special discount when you use borg and "you're an expert" https://www.rsync.net/products/borg.html https://www.rsync.net/products/borg.html We're looking to replace our self-written borg backup scripts with https://torsion.org/borgmatic/ https://torsion.org/borgmatic/ which is a wrapper around borg.

n8henrie 4y ago
TIL! Same for restic: https://www.rsync.net/products/restic.html https://www.rsync.net/products/restic.html
- stavros 4y ago
  As far as I can tell (as a years-long user of this), this is a generic "here's a server that supports SCP, go nuts" discount. You don't have to use it with Borg or Restic, they're just supported.
  - asymmetric 4y ago
    Notably, you can't reformat the drives to another filesystem (say btrfs)
    - stavros 4y ago
      Can you do that over SCP in general, without a full shell?
berkut 4y ago
What are the practicalities of the '680 GB Minimum Order'? I would have expected you pay by how much you use (hence the price per GB), but the minimal order (the submission form however won't let you submit less than 100GB, so it's a bit confusing there's a mis-match?) seems slightly at odds with that. (The FAQ doesn't seem to answer that)
- andreareina 4y ago

aborsy 4y ago

Borg is very good. The V2 repository format will bring in a lot of improvements, particularly in cryptography. Anyone knows when 2.0 will be out of beta, and stable?

m3nu 4y ago
Likely next year after 1-2 RCs. It's at beta4 currently.

nine_k 4y ago

If you prefer a similar approach, but as a single compiled binary, there's Restic: https://github.com/restic/restic https://github.com/restic/restic Update: yet another take on basically the same approach, also as a self-contained binary: https://github.com/kopia/kopia https://github.com/kopia/kopia

kova12 4y ago
I recall trying to use restic instead of borg a couple years ago, and some major feature was unavailable. I don't recall what is was, I think it was compression, which made archives quite large, and required larger instances for backup.
- deleted 4y ago
  [deleted]
- anotherevan 4y ago
  It probably was compression. The good news is compression is now available with Restic!
RockRobotRock 4y ago
Does anyone have a take on Kopia vs Restic?
- btschaegg 4y ago
  A major factor why I wouldn't want to use Kopia (I looked into it) ist that it is opinionated with regards to how your system is set up (old-school unix FS layouts in a "pets, not cattle" way). It assumes the location of config files and does not allow you to change the backup path that's stored in a snapshot's metadata. That's bad if you want to use it 1) on NixOS (I don't want backup configs laying around in `~/.config`). As Indy famously said: "That belongs in a Nix expression!" Edit: On a second note: Why have mandatory config files at all? I don't have an issue with having the option or it being the default, but for my use case being able to specify the whole repository config via arguments sounds considerably more sane. 2) with ZFS snapshots (yes, I'm backing up `/path/to/dataset/.zfs/snapshot/<timestamp>/foo/bar`, but that should not be its path in the metadata!) OTOH, it seems to have the upside that you can apparently alter snapshots after the fact more easily (e.g. if you find out you shouldn't have backed up that gigantic VM image you just moved somewhere temporarily). I leave the decision on whether this is a footgun or not to you. And to be clear: The ZFS snaphot thing is also a pain with Restic, too. You can hack around it somewhat better with something like systemd-nspawn, but it really shouldn't be that hard.

SoftTalker 4y ago

Maybe a bit off topic, but what is a good utility for "imaging" a linux system. I have a task to reprovision a system but we want to keep a complete backup of the current system so that it's possible to restore completely as if it were never touched. This is more than just data backup as we would need need to recover disk partitions/LVM metadata, boot records, etc. as well as all the data itself.

av8avenger 4y ago
Take a look at Clonezilla. Used it many times for the exact same purpose. You could run it either on a running system or use the live iso they provide. https://clonezilla.org/ https://clonezilla.org/
eointierney 4y ago
Clonezilla is awesome, fast, stable, flexible, and reliable, from the Taiwan Supercomputing Centre. I used it lots over a decade ago to manage Mac, Windows, and Linux workstations and servers.
akerl_ 4y ago
Do you need to do this once, or 10 times, or 1000 times? How big are the disks? The most boring answer is "connect the disks to something else and use `dd` to copy the full blocks from start to finish into a file".
- SoftTalker 4y ago
  For this specific need, just once. Disk is 1TB but only about 350GB used. 'dd' would have been my thought as well, I've heard of Clonezilla also but never used it and not sure it's really doing anything appreciably different. I like the idea of 'dd' because I have a very clear mental picture of what it does. Just wasn't sure there was something else I might want to look at.
  - dividuum 4y ago
    If you need multiple version of such a disk image, a tool like restic (or I guess borg too, not sure?) can also compress what's provided to it via stdin. So you'd dd directly into restic and it will delta compress to earlier backups.

vbezhenar 4y ago

Can someone suggest an approach to backup container environment? E.g. running inside Kubernetes. As I see it: I write some kind of configuration. someproject-db is a deployment which runs a postgres db. Tool should connect to this DB, issue some kind of pg_backup command, capture output, retrieve some metadata about previous backup from S3, compute difference with previous run, compress that difference and store it to S3. anotherproject is a deployment which runs an sqlite db. Tool should do the same but with sqlite-specific commands. yetanotherproject-data is a pvc which has attached pv. Tool should find pod which mounted this volume, exec into that pod and retrieve pv data, again find different and store it to S3. Of course things should be configurable. Like store difference every 15 minutes, store complete backup every week and so on. I'm fine with manual recover and with manual configuration (I just don't want to write and test all the scripts myself). What I don't want is some kind of magic tool which will backup the entire cluster, etcd and my grandparents automatically in some magic way only for $50k/cpu core.

aborsy 4y ago
Database and VM snapshot and backup can be tricky. My suggestion is using ZFS.
- mekster 4y ago
  Doesn't zfs kind of solve all the backup problems alone? Technically, no other backup tools can beat it as being the filesystem itself, it knows more than any external tools can know, like instantly know what file got changed over time without scanning the entire tree. I use Borg as backup of backup (zfs snapshots), so I'll be having multiple implementations of backups (also both are on different remote location) just to be on the safe side. I don't use any other fancier ones as I don't like risking data on less reliable tools.
  - m3nu 4y ago
    Remote ZFS replication kinda does. But the offsite backup wouldn't be encrypted and not everyone is using ZFS. So it's not for all situations.

dang 4y ago

Related: Deduplicating Archiver with Compression and Encryption - https://news.ycombinator.com/item?id=27939412 https://news.ycombinator.com/item?id=27939412 - July 2021 (71 comments) BorgBackup: Deduplicating Archiver - https://news.ycombinator.com/item?id=21642364 https://news.ycombinator.com/item?id=21642364 - Nov 2019 (103 comments)

ThePhysicist 4y ago

Have been using Borg for many years now, it saved me several times already when I accidentally deleted stuff I realized I still needed later on. What's great is that you can just mount your backup repository as a FUSE filesystem, Borg then gives you a directory structure containing all your backups over time. Personally I use dates to name my backups, e.g. 2022-10-11, so when I need to restore something from a specific date I just go to the appropriate folder and extract it.

mgbmtl 4y ago
Another nice thing with fuse is that I can let my users (devs) mount the backups with Borg, so that they can restore specific files that they need without asking me. And the mount can restrict which directories to expose, so they only see the files they need. (I wrote a small wrapper which can be run with sudo)

andrewchambers 4y ago

I am the author of bupstash - https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash which has many advantages over borg in my biased opinion (like air gapped decryption keys and better performance). Feel free to check it out.

g_p 4y ago
Thanks for your work on bupstash. To second this, there are some threat models that are often overlooked with backups. Separating the encryption and restore keys makes sense, and is a design pattern few of the modern new backup tools offer - bupstash and duplicacy are the two that spring to mind which support asymmetrically encrypted backups. The recent lastpass issue shows the importance of keeping the backup decryption keys offline and inaccessible. A headless server with backup script can upload backups, but if someone can access the script and backups, they have the golden goose (users' vaults). Most other backup tools don't give you the ability to separate out creating and retrieving backups. Separate moan - very few of the backup tools that use s3 style buckets give a proper description of the IAM security model you should use - I like to understand what object prefixes are used, and what the absolute minimum I require is, as I like to start from minimum privilege - PutObject, GetObject on an index or config file, and nothing else if I can avoid it! Hopefully we see more focus on backup security after lastpass.
dvzk 4y ago
I was about to mention this. It’s hard for me to take any symmetric or password-based backup tools seriously. User chosen passwords are a disaster for offline encryption. Most tools also use related key input for (non-)authenticated encryption and decryption. A server shouldn’t be able to decrypt backups and authentically encrypt new backups.
arde 4y ago
Very interesting, I like its design better. Could you comment on whether bupstash supports ACLs (POSIX, xattr, etc.)? According to issue #25 it does, but I couldn't find documentation on how to exclude it when restoring, viewing it, etc.

m3drano 4y ago

I recommend using borgmatic to ease the management of Borg backups.

nov21b 4y ago

Just started using the append only feature to prevent a potential hacker from wiping out backups that live on a remote ssh server. Combined with restricted ssh access this can be made quite secure. I also tested writing backups to my Android phone (as a backup target) using Termux and Wireguard, worked flawlessly with a bit of tuning (keeping the vpn alive)

pkulak 4y ago
Append only modes are brilliant. Is there an easy way to hook into something like Glacier Deep Archive? That would be super cost effective.
- nov21b 4y ago
  Not without akward hacks. [0] I backup to a Hetzner storage box and a Raspberry Pi at home. [0] https://github.com/borgbackup/borg/issues/1872 https://github.com/borgbackup/borg/issues/1872

lyu07282 4y ago

Its pretty great, I just wished it somehow integrated with rclone, I wish I could directly backup to google drive with it.

beagle3 4y ago
restic has a very similar feature set (it has compression now), and is well integrated with rclone.

tschuehly 4y ago

I'm searching for an alternative to OneDrive for my personal files, as I'm already paying for 1TB S3 Storage I wanted to use that as Backend. What tool could I use that has a windows client, android and ios?

phyzome 4y ago

I've been very happy with it. I switched away from Tarsnap because it was far too slow on restores. BorgBackup has slightly different security semantics, but similarly supports the notion of append-only credentials. And borgbase.com has been a good place to host my backups. Really nice onboarding flow.

lighttower 4y ago
I've been using it with Rsync.net also very happy.
andreareina 4y ago
Borg has an append-only mode, but no append-only credentials as far as I could find (by necessity given the symmetric encryption I believe). That means the key still needs to be protected, which makes secure automated backups a bit of a pickle.
- g_p 4y ago
  If you use a remote Borg server over SSH, you can set up your authorized_keys file to enforce append-only mode on the server by defining the command to be run at login, and reject Borg requests by that SSH key which try to purge or remove data. I believe this is roughly what borgbase does to implement their own backup protection features.
  - andreareina 4y ago
    Yeah I suppose that's a fair interpretation of the authorized_keys setup.
- phyzome 4y ago
  Ah, yes, my apologies -- it's done via a connection mode.

dawnerd 4y ago

Borg is really solid and I had to unfortunately use it to restore data a bit ago and it went very smoothly. I’m bad with testing backups so I was a bit nervous.

gorgoiler 4y ago

I’ve been using Borg for the last three years to replicate on-site ZFS snapshots as Borg archives on rsync.net. My local ZFS is plaintext on LUKS volumes and the remote Borg archives are encrypted with borg. Would I be better off using ZFS native encryption and replicating the encrypted stream without giving a key to the rsync.net side? I worry that one false move would mean I would have to do a full backup again. I think I kind of feel more comfortable having borg as a second technology in the system, too, so that my restore can’t be broken by a ZoL bug. Doing test restores with Borg is also nice because I don’t need a ZFS system to do the restore — restoring to macOS works just fine for example. Funnily enough, I’ve never had to prune / ruler function any Borg archives. I just don’t have enough churn in my data to need to do it. Everything change is there in full. It works so well. Thank you everyone involved for donating your time and ideas to providing these tools and services.

rsync 4y ago
"Would I be better off using ZFS native encryption and replicating the encrypted stream without giving a key to the rsync.net side?" I think you mean you are using borgs "repo key" method where the key to the repo is stored, remotely, inside the repo, correct ? Just to clarify for others: That repo-stored key has a passphrase so this method still keeps all of your files encrypted in a way that nobody at rsync.net can access. FWIW, I use this method personally - I like the idea of storing the key with the repo and unlocking it with a passphrase.
- gorgoiler 4y ago
  Thanks for responding and for providing such a great service. I meant dropping borg altogether and using zfs send/receive to sync my data to rsync.net. As I understood it, if my dataset is encrypted then rsync.net only sees the name of the snapshot and the encrypted data. I’m not sure this would be a great solution for my most common restore use case though: fetching a single file back with borg while I’m at a third location that is remote to both rsync.net and my ZFS host. The fact that I can do that with just Borg and rsync.net is incredibly useful (yes, I travel with my keys!)

unaindz 4y ago

I tried a few backup tools and https://github.com/andrewchambers/bupstash https://github.com/andrewchambers/bupstash is my favorite by far but it's not that well known. It was pretty fast already and recently got multithread support. It has been the only thing usable for backing up a few TB in a raspberry for performance reasons. Keep in mind it's relatively new and the author does not yet recommend to use in production as the only backup solution.

nextaccountic 4y ago
does this also backup mtime and ctime timestamps?
- andrewchambers 4y ago
  Yes and it can restore mtime and atime - but not ctime.
  - nextaccountic 4y ago
    What can it do with ctime? Maybe it could provide a virtual read only filesystem, mounting the backup at a specific date, using the original atime, mtime and ctime
    - andrewchambers 4y ago
      It can provide diffs against ctime so you can see if files have been modified and then restored.

layer8 4y ago

Can anyone recommend a (Linux CLI) backup tool for incremental encrypted backups via SCP that allows a verification run against the current filesystem state while excluding a configured list of paths from the verification (for some frequently changing files)? I’m currently using duplicity, which is missing that last feature (without post-processing its verification output).

stavros 4y ago
Restic can do all that, I believe.
- deleted 4y ago
  [deleted]

mikob 4y ago

How does this compare to rsnapshot?

ThomasWaldmann 4y ago
Not much. rsnapshot is based on rsync and hardlinks. So, you'll get some whole-file deduplication for unmodified files. No compression, no encryption/authentication, no way to verify if a backup is still ok. Also, if you rename a file or folder, that hardlink-based dedup will not work, because it is based on files having the same path. Also, if you change just 1 bit in a huge file, it will add the whole new file because the dedup only works on whole files. borg does variable size chunk based deduplication. if a big file changed just a bit, your backup repo will also just grow a bit when backing up that file. also you get compression, encryption, authentication and the ability to check backups if they are ok still. also, borg won't create a gazillion of files/hardlinks in the destination filesystem, but way fewer files (each about 500MiB big by default). borg also archives ACLs, xattrs, flags.

tut-urut-utut 4y ago

> Easy installation on multiple platforms: Linux, macOS, BSD, ... No Windows :( Luckily there are capable alternatives to Borg, including Restic and Kopia.

bilekas 4y ago

Hmmm I like this a lot actually, but as an encrypted file repo for sensitive files/repos themselves. This is quite clever, don't think I've seen all these features together before.

Brian-Puccio 4y ago

I’ve been using BorgBackup since it was Attic. Been thrilled with it.

siraben 4y ago

Absolutely love Borg, have been using it for several years. Cross-platform (macOS and Linux) backups, deduplication, encryption and compression, FUSE mounting, backing up over SSH (works great in combination with Tailscale and can be interrupted/resumed). Pruning backups with rules is also easy, and sometimes I accidentally back up really large folders in which case Borg lets you delete it from all the backups.

avodonosov 4y ago

Is Borg significantly better than rsync to a btrfs or zfs volume, and then snapshotting that volume?

pQd 4y ago

borg is great. we've been using it for the past 3 years to archive hundreds of file-level backups of servers, database dumps and VM images. average size of each borg repo is few GB but there are few outliers up to few hundreds of GB. most of backups are done daily, with 7-24 past days preserved in borg archive. borg repos are verified, copied to external disks, verified again and rotated offline each week. borg replaced https://rdiff-backup.net/ https://rdiff-backup.net/ for us and gave: * nice speedup of backups/backup tests, * decent saving in the disk space thanks to compression and deduplication, * decreased backup replication time [ borg repo tends to have much less, larger files compared to rdiff which has in its repo at least as many files as your source data; rsync likes it ]. to finish backups in a reasonable time we had to parallelize backup gathering [ each server / vm goes to separate borg repo; this limits a failure domain in case of corrupted repo, but denies us benefit of deduplication on larger scale - across servers ] and borg archiving. without that - we would be a limited by a single cpu core performance [ borg is not multithreaded yet ]. it's worth testing the backups - we're doing it each day by using borg's repo self test and by extracting few key files and checking their checksums and content... just in case. echoing other comments - https://kopia.io/ https://kopia.io/ looks interesting but we have not tried it yet.

AnonC 4y ago

Any pointers on running borg or restic on a Synology NAS running DSM 7 (to backup the NAS to another destination)? The Synocommunity package for Borg generally tends to be lagging behind the official releases by months. And Docker seems to be a bit complex too (since the NAS I have directly doesn’t support it).

leaving 4y ago

I don't understand why creators of projects like this obscure the basics. Important things you need to tell me: (1) The file I download is a command-line executable. I have to chnmod +x before I can use it. Why not share that small fact? (2) How do I create a simple repository *with no encryption*. Yes, I want to start there. Why do you think I want to get a PhD in your project before trying it? No, I don't want to start by creating encryption keys; that comes later. Start with the basic, basic, basic instructions. (3) Why did I export that environment variable? I never used it, did I? Not so far as I can tell. Is that basic? No, it is not. (4) What does recreate do? Is it necessary? "borg rcreate --help" does *not* provide any help on the encryption options. It just provides an error. Seriously, what is wrong with you people? Can you not imagine what it is like to come into an open source project with no experience and no knowledge of the matter whatsoever? Is it machismo? Is it elitism? What, in the name of God, prevents you from saying, "This is my project. It does this. This is the "hello world" method of trying it. Here are some cool options once you have the basics. It happens almost all the time. I seriously don't know what is wrong with you. Edit: I had to search forever to find the keyword "none" that you can put after the "-e" parameter. Found it by trial and error. It's in the docs, but, Jesus, people, can you not imagine for a moment that someone might be testing this on not the latest hardware and want to see how it runs without the complexity encryption first? You absolutely blow my mind. I don't know if you're putting me on, here. Do you not know how to communicate?

dikei 4y ago
You're welcome to read the Quick Start docs, it's pretty short https://borgbackup.readthedocs.io/en/1.2-maint/quickstart.html https://borgbackup.readthedocs.io/en/1.2-maint/quickstart.ht...
crossroadsguy 4y ago
I agree. Borg (and restic as well - though it fares better than borg in usability), even after tinkering with it for years, remained essentially useless for me until Vorta (a borg gui) came along. https://vorta.borgbase.com https://vorta.borgbase.com is awesome. Simple and awesome. I wish it gains more exposure.
sigtstp 4y ago

jve 4y ago

Dunno if this 1 year old comment is relevant, but to whomever it may concern, it may be useful to research backup software stability & reliability: https://news.ycombinator.com/item?id=29210222 https://news.ycombinator.com/item?id=29210222

asymmetric 4y ago

Is there anything like zfs.rent for btrfs? I would like to use my filesystem’s facilities for backing rather than a whole piece of software on top, but I haven’t found any managed storage solutions yet.

INTPenis 4y ago

I used borg for years, never had any issue. Even though there were scares of bad versions that could corrupt your archives. Never happened to my personal backups of a few hundred Gs at most. But I had to switch to restic because the S3 support was too good. Off-site backups in one command, instead of having to maintain my borg repos on NAS and external disks.

potamic 4y ago

Can you use this for a full system backup or is it more suited towards backing up individual files?

qwertygerty 4y ago

Unlike some of the negative comments seen, I'd like to extend my gratitude to the developers for this work! I've recently decided to review my backup scheme, and have been looking at btrfs. Anyone out there doing a btrfs+borg combo? What is your scheme? I've found this as an example, but it's quite a bit more intense than what I need... https://mutschler.dev/stuff/backup/ https://mutschler.dev/stuff/backup/